2022-04-07T09:38:22.822 INFO:root:teuthology version: 1.1.0-3dd79763 2022-04-07T09:38:22.942 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging 2022-04-07T09:38:22.943 INFO:teuthology.lock.ops:Updating [ubuntu@smithi138.front.sepia.ceph.com]: reset os type and version on server 2022-04-07T09:38:22.944 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-04-07T09:38:22.961 INFO:teuthology.lock.ops:Node 'ubuntu@smithi138.front.sepia.ceph.com' reimaging is complete 2022-04-07T09:38:22.961 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi197.front.sepia.ceph.com' reimaging 2022-04-07T09:38:22.962 INFO:teuthology.lock.ops:Updating [ubuntu@smithi197.front.sepia.ceph.com]: reset os type and version on server 2022-04-07T09:38:22.962 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server 2022-04-07T09:38:22.982 INFO:teuthology.lock.ops:Node 'ubuntu@smithi197.front.sepia.ceph.com' reimaging is complete 2022-04-07T09:38:23.167 INFO:teuthology.provision.fog.smithi138:Scheduling deploy of ubuntu 20.04 2022-04-07T09:38:23.180 INFO:teuthology.provision.fog.smithi197:Scheduling deploy of ubuntu 20.04 2022-04-07T09:38:23.473 INFO:teuthology.orchestra.console:Power off smithi138 2022-04-07T09:38:23.487 INFO:teuthology.orchestra.console:Power off smithi197 2022-04-07T09:38:35.952 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-04-07T09:38:35.957 INFO:teuthology.orchestra.console:Power off for smithi197 completed 2022-04-07T09:38:36.054 INFO:teuthology.orchestra.console:Power on smithi138 2022-04-07T09:38:36.060 INFO:teuthology.orchestra.console:Power on smithi197 2022-04-07T09:38:40.316 INFO:teuthology.orchestra.console:Power on for smithi197 completed 2022-04-07T09:38:40.417 INFO:teuthology.provision.fog.smithi197:Waiting for deploy to finish 2022-04-07T09:38:44.423 INFO:teuthology.orchestra.console:Power on for smithi138 completed 2022-04-07T09:38:44.524 INFO:teuthology.provision.fog.smithi138:Waiting for deploy to finish 2022-04-07T09:43:36.452 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-07T09:43:37.029 INFO:teuthology.provision.fog.smithi197:Node is ready 2022-04-07T09:43:37.079 INFO:teuthology.orchestra.run.smithi197.stdout:smithi197.front.sepia.ceph.com 2022-04-07T09:43:37.087 INFO:teuthology.orchestra.run.smithi197.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197 2022-04-07T09:43:37.290 INFO:teuthology.provision.fog.smithi197:Deploy complete! 2022-04-07T09:43:40.279 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-07T09:43:40.771 INFO:teuthology.provision.fog.smithi138:Node is ready 2022-04-07T09:43:40.777 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138.front.sepia.ceph.com 2022-04-07T09:43:40.783 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138 smithi138.front.sepia.ceph.com smithi138 2022-04-07T09:43:40.995 INFO:teuthology.provision.fog.smithi138:Deploy complete! 2022-04-07T09:43:41.192 INFO:teuthology.lock.ops:Checking smithi138.front.sepia.ceph.com 2022-04-07T09:43:41.193 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-07T09:43:41.212 INFO:teuthology.lock.ops:Checking smithi197.front.sepia.ceph.com 2022-04-07T09:43:41.212 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-07T09:43:41.232 INFO:teuthology.lock.ops:Updating [smithi138.front.sepia.ceph.com]: set os type and version on server 2022-04-07T09:43:41.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-04-07T09:43:41.422 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2022-04-07T09:43:41.472 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="Ubuntu" 2022-04-07T09:43:41.472 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-04-07T09:43:41.473 INFO:teuthology.orchestra.run.smithi138.stdout:ID=ubuntu 2022-04-07T09:43:41.473 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE=debian 2022-04-07T09:43:41.473 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-04-07T09:43:41.473 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="20.04" 2022-04-07T09:43:41.473 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.ubuntu.com/" 2022-04-07T09:43:41.474 INFO:teuthology.orchestra.run.smithi138.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-04-07T09:43:41.474 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-04-07T09:43:41.474 INFO:teuthology.orchestra.run.smithi138.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-04-07T09:43:41.474 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_CODENAME=focal 2022-04-07T09:43:41.474 INFO:teuthology.orchestra.run.smithi138.stdout:UBUNTU_CODENAME=focal 2022-04-07T09:43:41.475 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2022-04-07T09:43:41.495 INFO:teuthology.lock.ops:Updating [smithi197.front.sepia.ceph.com]: set os type and version on server 2022-04-07T09:43:41.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi197.front.sepia.ceph.com' 2022-04-07T09:43:41.678 INFO:teuthology.orchestra.run.smithi197.stdout:x86_64 2022-04-07T09:43:41.685 INFO:teuthology.orchestra.run.smithi197.stdout:NAME="Ubuntu" 2022-04-07T09:43:41.685 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-04-07T09:43:41.686 INFO:teuthology.orchestra.run.smithi197.stdout:ID=ubuntu 2022-04-07T09:43:41.686 INFO:teuthology.orchestra.run.smithi197.stdout:ID_LIKE=debian 2022-04-07T09:43:41.686 INFO:teuthology.orchestra.run.smithi197.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-04-07T09:43:41.686 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_ID="20.04" 2022-04-07T09:43:41.686 INFO:teuthology.orchestra.run.smithi197.stdout:HOME_URL="https://www.ubuntu.com/" 2022-04-07T09:43:41.687 INFO:teuthology.orchestra.run.smithi197.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-04-07T09:43:41.687 INFO:teuthology.orchestra.run.smithi197.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-04-07T09:43:41.687 INFO:teuthology.orchestra.run.smithi197.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-04-07T09:43:41.687 INFO:teuthology.orchestra.run.smithi197.stdout:VERSION_CODENAME=focal 2022-04-07T09:43:41.688 INFO:teuthology.orchestra.run.smithi197.stdout:UBUNTU_CODENAME=focal 2022-04-07T09:43:41.688 INFO:teuthology.lock.ops:Updating smithi197.front.sepia.ceph.com on lock server 2022-04-07T09:43:41.787 INFO:teuthology.dispatcher.supervisor:Running job 6780575 2022-04-07T09:43:41.788 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3dd797633f37e2d7e2c4ae3ef9ac35a188c2028d/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2022-04-07_05:07:02-rgw-pacific-distro-default-smithi/6780575 --name teuthology-2022-04-07_05:07:02-rgw-pacific-distro-default-smithi --description rgw/multisite/{clusters frontend/civetweb ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub tasks/test_multi valgrind} -- /home/teuthworker/archive/teuthology-2022-04-07_05:07:02-rgw-pacific-distro-default-smithi/6780575/orig.config.yaml 2022-04-07T09:43:41.791 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2022-04-07_05:07:02-rgw-pacific-distro-default-smithi/6780575 2022-04-07T09:43:41.792 INFO:teuthology.dispatcher.supervisor:Job PID: 30477 2022-04-07T09:43:41.792 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2022-04-07T16:14:09.918 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing... 2022-04-07T16:14:10.077 INFO:teuthology.kill:Killing Pids: {30477} 2022-04-07T16:14:11.200 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1'] 2022-04-07T16:14:11.201 INFO:teuthology.task.internal:roles: ubuntu@smithi197.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1'] 2022-04-07T16:14:11.202 INFO:teuthology.misc:Compressing logs... 2022-04-07T16:14:11.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2022-04-07T16:14:11.382 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi197.front.sepia.ceph.com' 2022-04-07T16:14:11.735 INFO:teuthology.misc:Compressing logs... 2022-04-07T16:14:11.823 INFO:teuthology.misc:Compressing logs... 2022-04-07T16:14:19.250 INFO:teuthology.kill:No teuthology processes running 2022-04-07T16:14:20.310 INFO:teuthology.kill:Nuking machines: ['smithi138', 'smithi197'] 2022-04-07T16:14:21.282 INFO:teuthology.kill:2022-04-07 16:14:21,282.282 INFO:teuthology.nuke:targets: 2022-04-07T16:14:21.283 INFO:teuthology.kill: smithi138.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLWeO9/Aj/CzO8uZEDm3oVgCU7pbQa7NVHKKpR7dc9E7kUs0xdqbwRbg3OWCu5Sh5PLniZnY06WKfMuR3isC3yru2muY5mTzJr53i6j3SDiSQOeq72E9rqGqbkV/L3y3rxSRq1Ilxm6+gkKjsOJEBNyp9n6YkQlRv0QKttXFr+rWrRmBpiXDGkFaZpR7og69Y/L9XGpIht+Yj4VO7vkg54blQ3/8jaxmUbSMBATUb/CrmdhoMoXY8nM9hUqqogaMVXAh9DEGKm80MxAfcCYtUdd6Fc83riux+6CgAsnGv+rT+0ahIgGQc4uKhMqkjWcywqo4Wwxm9jP/JAygR5SLYH8ampfz8NbOLDH1C7ATiR6+V70kgrnqWvntyuGVJZHXa20/gpQx6y8gJ3mfetu122Jk4lLSUinub8dbj3snoNhTsOr1Zq/ITuTDTuxOJBUH24nnXEt5XkFpJfelDiYz09LcJdoEZMRIfzBHEmKK2YmObb/l/H04+FdxGooi0CqVU= 2022-04-07T16:14:21.284 INFO:teuthology.kill: smithi197.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZLC1YPOdBLmmoths+e2CgP9DFouBbwDZJMdquOwUObcEgLJY2cqoC36UzF2fVsRq6Ny6IbM2y5QILUWblkpHbGwl7gdpiNu6E35vlKEeRxrP82P2AGTz041TFRYjb2d4H11FJZlmrVGU83i0Tm75PCnS5eyAjQeEPQUWkm6WUZLQtMv149iFK8lX7G/LMelMLaibnBTrt6F4swv4KonORlVxs78BSUab2uncNFlXfd8tpLnRG1v5iz2t8Txtzw0cKgn+N7zRemVIIIlWHNo1MHTJYWyG0TYxqFCXT7gwNkaFdV+yVEps8jPLWPVYVwIHXH5vuRajnes8pM40sOT51me1RyUnci+iaQ+6fbdauO3iNgHABHO0sO8pAfPjwtiMfKDzfKDErreTel8X/tiNMcP2+Km63wf2DpLy3v0HdylX2wFkKwNQ5lP3DW4YscfDuG5zcYhyyGmNvgCZbZw/E1JWhjMpdEpckvsqWPwYGlPuHrqIO1ys1tiF0IJnfeZ0= 2022-04-07T16:14:21.308 INFO:teuthology.kill:2022-04-07 16:14:21,308.308 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-07T16:14:21.311 INFO:teuthology.kill:2022-04-07 16:14:21,311.311 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-07T16:14:21.381 INFO:teuthology.kill:2022-04-07 16:14:21,380.380 INFO:teuthology.orchestra.console:Power off smithi197 2022-04-07T16:14:21.395 INFO:teuthology.kill:2022-04-07 16:14:21,395.395 INFO:teuthology.orchestra.console:Power off smithi138 2022-04-07T16:14:29.748 INFO:teuthology.kill:2022-04-07 16:14:29,748.748 INFO:teuthology.orchestra.console:Power off for smithi197 completed 2022-04-07T16:14:29.755 INFO:teuthology.kill:2022-04-07 16:14:29,754.754 INFO:teuthology.orchestra.console:Power off for smithi138 completed 2022-04-07T16:14:29.898 INFO:teuthology.kill:2022-04-07 16:14:29,897.897 INFO:teuthology.lock.ops:unlocked: smithi197.front.sepia.ceph.com 2022-04-07T16:14:29.929 INFO:teuthology.kill:2022-04-07 16:14:29,929.929 INFO:teuthology.lock.ops:unlocked: smithi138.front.sepia.ceph.com 2022-04-07T16:16:30.200 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2022-04-07T16:16:30.295 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi138 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-04-07_15:19:12-fs-wip-vshankar-testing-55110-20220407-173953-testing-default-smithi/6780955 2022-04-07T16:16:30.295 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi197 but it was locked by another job: /home/teuthworker/archive/vshankar-2022-04-07_15:19:12-fs-wip-vshankar-testing-55110-20220407-173953-testing-default-smithi/6780955