2021-11-03T14:56:36.159 INFO:root:teuthology version: 1.1.0-c56135d1
2021-11-03T14:56:36.276 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi012.front.sepia.ceph.com' reimaging
2021-11-03T14:56:36.277 INFO:teuthology.lock.ops:Updating [ubuntu@smithi012.front.sepia.ceph.com]: reset os type and version on server
2021-11-03T14:56:36.278 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server
2021-11-03T14:56:36.296 INFO:teuthology.lock.ops:Node 'ubuntu@smithi012.front.sepia.ceph.com' reimaging is complete
2021-11-03T14:56:36.296 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging
2021-11-03T14:56:36.297 INFO:teuthology.lock.ops:Updating [ubuntu@smithi110.front.sepia.ceph.com]: reset os type and version on server
2021-11-03T14:56:36.297 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2021-11-03T14:56:36.316 INFO:teuthology.lock.ops:Node 'ubuntu@smithi110.front.sepia.ceph.com' reimaging is complete
2021-11-03T14:56:36.507 INFO:teuthology.provision.fog.smithi012:Scheduling deploy of ubuntu 20.04
2021-11-03T14:56:36.511 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of ubuntu 20.04
2021-11-03T14:56:36.824 INFO:teuthology.orchestra.console:Power off smithi110
2021-11-03T14:56:36.862 INFO:teuthology.orchestra.console:Power off smithi012
2021-11-03T14:56:45.200 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-11-03T14:56:45.301 INFO:teuthology.orchestra.console:Power on smithi110
2021-11-03T14:56:49.337 INFO:teuthology.orchestra.console:Power off for smithi012 completed
2021-11-03T14:56:49.439 INFO:teuthology.orchestra.console:Power on smithi012
2021-11-03T14:56:53.665 INFO:teuthology.orchestra.console:Power on for smithi110 completed
2021-11-03T14:56:53.687 INFO:teuthology.orchestra.console:Power on for smithi012 completed
2021-11-03T14:56:53.767 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish
2021-11-03T14:56:53.788 INFO:teuthology.provision.fog.smithi012:Waiting for deploy to finish
2021-11-03T15:02:13.449 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-03T15:02:14.026 INFO:teuthology.provision.fog.smithi110:Node is ready
2021-11-03T15:02:14.032 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com
2021-11-03T15:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110
2021-11-03T15:02:14.283 INFO:teuthology.provision.fog.smithi110:Deploy complete!
2021-11-03T15:03:29.298 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-11-03T15:03:29.841 INFO:teuthology.provision.fog.smithi012:Node is ready
2021-11-03T15:03:29.847 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012.front.sepia.ceph.com
2021-11-03T15:03:29.897 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12 smithi012.front.sepia.ceph.com smithi012
2021-11-03T15:03:30.071 INFO:teuthology.provision.fog.smithi012:Deploy complete!
2021-11-03T15:03:30.257 INFO:teuthology.lock.ops:Checking smithi012.front.sepia.ceph.com
2021-11-03T15:03:30.258 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-03T15:03:30.278 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com
2021-11-03T15:03:30.279 INFO:teuthology.lock.ops:New key found. Updating...
2021-11-03T15:03:30.299 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: set os type and version on server
2021-11-03T15:03:30.300 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com'
2021-11-03T15:03:30.476 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64
2021-11-03T15:03:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu"
2021-11-03T15:03:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-11-03T15:03:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu
2021-11-03T15:03:30.483 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian
2021-11-03T15:03:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-11-03T15:03:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="20.04"
2021-11-03T15:03:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/"
2021-11-03T15:03:30.484 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-11-03T15:03:30.485 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-11-03T15:03:30.485 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-11-03T15:03:30.485 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=focal
2021-11-03T15:03:30.485 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=focal
2021-11-03T15:03:30.486 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server
2021-11-03T15:03:30.505 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server
2021-11-03T15:03:30.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com'
2021-11-03T15:03:30.758 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64
2021-11-03T15:03:30.764 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Ubuntu"
2021-11-03T15:03:30.765 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-11-03T15:03:30.765 INFO:teuthology.orchestra.run.smithi110.stdout:ID=ubuntu
2021-11-03T15:03:30.765 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE=debian
2021-11-03T15:03:30.765 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-11-03T15:03:30.766 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="20.04"
2021-11-03T15:03:30.766 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.ubuntu.com/"
2021-11-03T15:03:30.766 INFO:teuthology.orchestra.run.smithi110.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-11-03T15:03:30.766 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-11-03T15:03:30.766 INFO:teuthology.orchestra.run.smithi110.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-11-03T15:03:30.767 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_CODENAME=focal
2021-11-03T15:03:30.767 INFO:teuthology.orchestra.run.smithi110.stdout:UBUNTU_CODENAME=focal
2021-11-03T15:03:30.767 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server
2021-11-03T15:03:30.862 INFO:teuthology.dispatcher.supervisor:Running job 6481390
2021-11-03T15:03:30.863 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2021-11-03_14:52:52-rgw-wip-cbodley-testing-distro-basic-smithi/6481390 --name cbodley-2021-11-03_14:52:52-rgw-wip-cbodley-testing-distro-basic-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{ubuntu_latest} tasks/test_multi} -- /home/teuthworker/archive/cbodley-2021-11-03_14:52:52-rgw-wip-cbodley-testing-distro-basic-smithi/6481390/orig.config.yaml
2021-11-03T15:03:30.866 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2021-11-03_14:52:52-rgw-wip-cbodley-testing-distro-basic-smithi/6481390
2021-11-03T15:03:30.867 INFO:teuthology.dispatcher.supervisor:Job PID: 11549
2021-11-03T15:03:30.867 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-11-04T03:04:25.072 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-11-04T03:04:25.244 INFO:teuthology.kill:Killing Pids: {11549}
2021-11-04T03:04:26.495 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1']
2021-11-04T03:04:26.496 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']
2021-11-04T03:04:26.497 INFO:teuthology.misc:Compressing logs...
2021-11-04T03:04:26.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com'
2021-11-04T03:04:26.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com'
2021-11-04T03:04:27.062 INFO:teuthology.misc:Compressing logs...
2021-11-04T03:04:27.158 INFO:teuthology.misc:Compressing logs...
2021-11-04T03:04:39.121 INFO:teuthology.kill:No teuthology processes running
2021-11-04T03:04:40.352 INFO:teuthology.kill:Nuking machines: ['smithi012', 'smithi110']
2021-11-04T03:04:41.363 INFO:teuthology.kill:2021-11-04 03:04:41,363.363 INFO:teuthology.nuke:targets:
2021-11-04T03:04:41.364 INFO:teuthology.kill:  smithi012.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIa25VPzmWb+2nJzBB0wFiMPXyfY4HeU+km/6VeeJlPLfACt/mUqGocUCsTJhljfwbx5uJ/LbZyB/43TmS06wjEbGZaBQBv8Lhy/6aogZS06vTgFvlVQGCztJITltLf/3u9xxGsmpeueE9POUyZHE//qxSWXbB5fWzosPIFccMVKnPGnvBaa/wZblBGMQaj9KlkUZAMwFJJiCnD5qj2bc1JSI92e96xJk8g2vpd4X/IeYZr2pcKmWp+G1z1Fp9CGoazfqf74htR6RldTVlWRVPFyzlVvSm5AMU5Oq2sflQqScIX7+NUCy1jtoQFzJWQSpXzClPr5+nlwZi28Fnvn/PKOlrMYJjeMkWckYDuEKbFrdRM/iDthSlAk7AAR08jW9zhpHt/fdbFEe4LdgAQ4S7WRfdk1zg5k1LqVkFpBRAWw0jpl9r70bYw+PKFOxpxP2X0f0i3vcONAYUUz5+eB93XithJCs/xPci6CcJNZhsxNxJ/144FJfnBYEujCKuaI8=
2021-11-04T03:04:41.364 INFO:teuthology.kill:  smithi110.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDxVZlibHDg4nbGCTRgMYaetagq6hULRWSvycs1+AAL9W/sanx31PNguwQOCSfGH6t/T2oEzINnFP9XQSnDsmTHeyF/VF7sCegryNVwQ3yqkxVv2I6V4B2KfvdgsQWef+ZrLVTZ/kVd1bnlFZThrTnXOFBZHukNXjAFp7zE6W+u9+3YYrEkptBkb4t4EcToNLf8PiHgeg/G6rwvkNjuuAvzrkt3uDDnFei6mWGWCpujOlwgg8KQRs8B/yc1NrMhOZqrxorETr7ZyfMDJGrWYrHgnnD8JpiBbZX1lgB3BAKHPMAQcM3aJh6ujBZsU/Y+mgwVKG/HM3Fb4sh1eLJwS3/sOGdGE0pudvhkYnXB+fRe7skLIbWB93lLJffeXsCCik6iAwAj6rvY31NSA+SZPEPa4VywnxnihF+vdcax1uZJKv4pSpTuWYPHnPEBft5l1hOmDhqfCJXOD/+wPKcDh0mC2bmUZLaG4HvoOW9SWW44p2L3rp+mWSAJtR92HXDStyc=
2021-11-04T03:04:41.390 INFO:teuthology.kill:2021-11-04 03:04:41,390.390 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-04T03:04:41.394 INFO:teuthology.kill:2021-11-04 03:04:41,394.394 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-11-04T03:04:41.472 INFO:teuthology.kill:2021-11-04 03:04:41,472.472 INFO:teuthology.orchestra.console:Power off smithi110
2021-11-04T03:04:41.486 INFO:teuthology.kill:2021-11-04 03:04:41,485.485 INFO:teuthology.orchestra.console:Power off smithi012
2021-11-04T03:04:53.955 INFO:teuthology.kill:2021-11-04 03:04:53,954.954 INFO:teuthology.orchestra.console:Power off for smithi110 completed
2021-11-04T03:04:53.960 INFO:teuthology.kill:2021-11-04 03:04:53,960.960 INFO:teuthology.orchestra.console:Power off for smithi012 completed
2021-11-04T03:04:54.103 INFO:teuthology.kill:2021-11-04 03:04:54,103.103 INFO:teuthology.lock.ops:unlocked: smithi012.front.sepia.ceph.com
2021-11-04T03:04:54.108 INFO:teuthology.kill:2021-11-04 03:04:54,108.108 INFO:teuthology.lock.ops:unlocked: smithi110.front.sepia.ceph.com
2021-11-04T03:06:54.421 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-11-04T03:06:54.516 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi012 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi/6482380
2021-11-04T03:06:54.517 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi110 but it was locked by another job: /home/teuthworker/archive/yuriw-2021-11-03_18:53:10-rados-wip-yuri5-testing-2021-11-02-0745-distro-basic-smithi/6482380