2022-08-26T16:42:50.088 INFO:root:teuthology version: 1.1.1.dev414+gc525e44d
2022-08-26T16:42:50.231 INFO:teuthology.lock.ops:Start node 'smithi057.front.sepia.ceph.com' reimaging
2022-08-26T16:42:50.233 INFO:teuthology.lock.ops:Updating [smithi057.front.sepia.ceph.com]: reset os type and version on server
2022-08-26T16:42:50.233 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server
2022-08-26T16:42:50.260 INFO:teuthology.lock.ops:Node 'smithi057.front.sepia.ceph.com' reimaging is complete
2022-08-26T16:42:50.260 INFO:teuthology.lock.ops:Start node 'smithi079.front.sepia.ceph.com' reimaging
2022-08-26T16:42:50.261 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: reset os type and version on server
2022-08-26T16:42:50.261 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server
2022-08-26T16:42:50.282 INFO:teuthology.lock.ops:Node 'smithi079.front.sepia.ceph.com' reimaging is complete
2022-08-26T16:42:50.479 INFO:teuthology.provision.fog.smithi057:Scheduling deploy of ubuntu 20.04
2022-08-26T16:42:50.495 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of ubuntu 20.04
2022-08-26T16:42:50.787 INFO:teuthology.orchestra.console:Power off smithi079
2022-08-26T16:42:50.802 INFO:teuthology.orchestra.console:Power off smithi057
2022-08-26T16:43:03.277 INFO:teuthology.orchestra.console:Power off for smithi079 completed
2022-08-26T16:43:03.284 INFO:teuthology.orchestra.console:Power off for smithi057 completed
2022-08-26T16:43:03.380 INFO:teuthology.orchestra.console:Power on smithi079
2022-08-26T16:43:03.389 INFO:teuthology.orchestra.console:Power on smithi057
2022-08-26T16:43:07.646 INFO:teuthology.orchestra.console:Power on for smithi057 completed
2022-08-26T16:43:07.747 INFO:teuthology.provision.fog.smithi057:Waiting for deploy to finish
2022-08-26T16:43:11.746 INFO:teuthology.orchestra.console:Power on for smithi079 completed
2022-08-26T16:43:11.848 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish
2022-08-26T16:48:15.771 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-08-26T16:48:16.279 INFO:teuthology.provision.fog.smithi057:Node is ready
2022-08-26T16:48:16.286 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057.front.sepia.ceph.com
2022-08-26T16:48:16.292 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57 smithi057.front.sepia.ceph.com smithi057
2022-08-26T16:48:16.510 INFO:teuthology.provision.fog.smithi057:Deploy complete!
2022-08-26T16:48:18.423 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-08-26T16:48:18.889 INFO:teuthology.provision.fog.smithi079:Node is ready
2022-08-26T16:48:18.897 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com
2022-08-26T16:48:18.905 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079
2022-08-26T16:48:19.135 INFO:teuthology.provision.fog.smithi079:Deploy complete!
2022-08-26T16:48:19.410 INFO:teuthology.lock.ops:Checking smithi057.front.sepia.ceph.com
2022-08-26T16:48:19.411 INFO:teuthology.lock.ops:New key found. Updating...
2022-08-26T16:48:19.433 INFO:teuthology.lock.ops:Checking smithi079.front.sepia.ceph.com
2022-08-26T16:48:19.433 INFO:teuthology.lock.ops:New key found. Updating...
2022-08-26T16:48:19.452 INFO:teuthology.lock.ops:Updating [smithi057.front.sepia.ceph.com]: set os type and version on server
2022-08-26T16:48:19.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com'
2022-08-26T16:48:19.697 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64
2022-08-26T16:48:19.704 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="Ubuntu"
2022-08-26T16:48:19.704 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2022-08-26T16:48:19.704 INFO:teuthology.orchestra.run.smithi057.stdout:ID=ubuntu
2022-08-26T16:48:19.705 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE=debian
2022-08-26T16:48:19.705 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2022-08-26T16:48:19.706 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="20.04"
2022-08-26T16:48:19.706 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://www.ubuntu.com/"
2022-08-26T16:48:19.706 INFO:teuthology.orchestra.run.smithi057.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-08-26T16:48:19.707 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-08-26T16:48:19.707 INFO:teuthology.orchestra.run.smithi057.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-08-26T16:48:19.707 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_CODENAME=focal
2022-08-26T16:48:19.707 INFO:teuthology.orchestra.run.smithi057.stdout:UBUNTU_CODENAME=focal
2022-08-26T16:48:19.709 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server
2022-08-26T16:48:19.735 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: set os type and version on server
2022-08-26T16:48:19.735 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com'
2022-08-26T16:48:19.945 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64
2022-08-26T16:48:19.995 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="Ubuntu"
2022-08-26T16:48:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="20.04.4 LTS (Focal Fossa)"
2022-08-26T16:48:19.996 INFO:teuthology.orchestra.run.smithi079.stdout:ID=ubuntu
2022-08-26T16:48:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE=debian
2022-08-26T16:48:19.997 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS"
2022-08-26T16:48:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="20.04"
2022-08-26T16:48:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://www.ubuntu.com/"
2022-08-26T16:48:19.998 INFO:teuthology.orchestra.run.smithi079.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-08-26T16:48:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-08-26T16:48:19.999 INFO:teuthology.orchestra.run.smithi079.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-08-26T16:48:20.000 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_CODENAME=focal
2022-08-26T16:48:20.000 INFO:teuthology.orchestra.run.smithi079.stdout:UBUNTU_CODENAME=focal
2022-08-26T16:48:20.000 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server
2022-08-26T16:48:20.172 INFO:teuthology.dispatcher.supervisor:Running job 6995287
2022-08-26T16:48:20.172 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_c525e44d32f89818c2b125dadbdafb95dc7366c8/virtualenv/bin/teuthology -v --owner scheduled_soumyakoduri@teuthology --archive /home/teuthworker/archive/soumyakoduri-2022-08-26_16:38:47-rgw:multisite-bucket-sync-testcases2-distro-default-smithi/6995287 --name soumyakoduri-2022-08-26_16:38:47-rgw:multisite-bucket-sync-testcases2-distro-default-smithi --description rgw:multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{ubuntu_latest} tasks/test_multi} -- /home/teuthworker/archive/soumyakoduri-2022-08-26_16:38:47-rgw:multisite-bucket-sync-testcases2-distro-default-smithi/6995287/orig.config.yaml
2022-08-26T16:48:20.178 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/soumyakoduri-2022-08-26_16:38:47-rgw:multisite-bucket-sync-testcases2-distro-default-smithi/6995287
2022-08-26T16:48:20.179 INFO:teuthology.dispatcher.supervisor:Job PID: 16225
2022-08-26T16:48:20.180 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-08-27T04:49:20.629 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2022-08-27T04:49:20.784 INFO:teuthology.kill:Killing Pids: {16225}
2022-08-27T04:49:23.168 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1']
2022-08-27T04:49:23.172 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']
2022-08-27T04:49:23.172 INFO:teuthology.misc:Compressing logs...
2022-08-27T04:49:23.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi057.front.sepia.ceph.com'
2022-08-27T04:49:23.532 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com'
2022-08-27T04:49:24.053 INFO:teuthology.misc:Compressing logs...
2022-08-27T04:49:24.194 INFO:teuthology.misc:Compressing logs...
2022-08-27T05:00:34.048 INFO:teuthology.kill:No teuthology processes running
2022-08-27T05:00:35.534 INFO:teuthology.kill:Nuking machines: ['smithi057', 'smithi079']
2022-08-27T05:00:36.792 INFO:teuthology.kill:/home/teuthworker/src/git.ceph.com_git_teuthology_c525e44d32f89818c2b125dadbdafb95dc7366c8/virtualenv/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.
2022-08-27T05:00:36.793 INFO:teuthology.kill:  from cryptography.hazmat.backends import default_backend
2022-08-27T05:00:36.793 INFO:teuthology.kill:2022-08-27 05:00:36,791.791 INFO:teuthology.nuke:targets:
2022-08-27T05:00:36.793 INFO:teuthology.kill:  smithi057.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYu3cqDxSqDee/G/QQgDsD1o1SrMif5IdyWK+wlNSKBuC1xBt0xNoP7Q1Tr9yVmSPG8BpQqFEbCM6nDKjSBhaSdpPrk5FbhLlaiNkg7FX7MEXFESs7Uys1MHdo4ZmXfQhRLwNU7dtenLKXM9q45QL7HSde2oKY72axav+9xzYiRmC7rFg1D9NJEzAFjkQeOKZsXXds2Ux3o8BlKtGRmgovSYu7i9a3KKMlxfVkePudjm+vEFnwWrJjHXKsnF1EdUZOi0kYYoMVXGELIhJDaPck/K4rpmJ7DOUrAdrSuoDquuqXnjfw9s0q9i6pTmkec/D+w5m6x7XST4gFSDx9yoiIWhwN/mFyrZUWfwDXuBRpL8sXy98+xVVc9E9w2lEKHTiP+s4oLnOFeMdVe3t8ipiqPmmLLCilmP92NB7cn6glrzIW4FmhA9jVlFMLkwXOarUe1ICEmd+MXM9+NSowKQTuou5u+0G7Mj0VI334u2Pk6VGRVn7tvPwp2EX9D+yaFYM=
2022-08-27T05:00:36.794 INFO:teuthology.kill:  smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3bNBQy+QDrqUiAvKcBuFuXinXMneASAwb0BgywpubViY8WblU9dVXmQiqlSHxykpfNz+jS1tfrlKxV/BVQq24=
2022-08-27T05:00:36.794 INFO:teuthology.kill:2022-08-27 05:00:36,792.792 INFO:teuthology.nuke:Checking targets against current locks
2022-08-27T05:00:36.858 INFO:teuthology.kill:2022-08-27 05:00:36,857.857 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-08-27T05:00:36.892 INFO:teuthology.kill:2022-08-27 05:00:36,891.891 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-08-27T05:00:36.955 INFO:teuthology.kill:2022-08-27 05:00:36,955.955 INFO:teuthology.orchestra.console:Power off smithi057
2022-08-27T05:00:37.007 INFO:teuthology.kill:2022-08-27 05:00:37,007.007 INFO:teuthology.orchestra.console:Power off smithi079
2022-08-27T05:00:45.378 INFO:teuthology.kill:2022-08-27 05:00:45,378.378 INFO:teuthology.orchestra.console:Power off for smithi079 completed
2022-08-27T05:00:45.543 INFO:teuthology.kill:2022-08-27 05:00:45,542.542 INFO:teuthology.lock.ops:unlocked: smithi079.front.sepia.ceph.com
2022-08-27T05:00:49.453 INFO:teuthology.kill:2022-08-27 05:00:49,452.452 INFO:teuthology.orchestra.console:Power off for smithi057 completed
2022-08-27T05:00:49.617 INFO:teuthology.kill:2022-08-27 05:00:49,614.614 INFO:teuthology.lock.ops:unlocked: smithi057.front.sepia.ceph.com
2022-08-27T05:02:50.008 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-08-27T05:02:50.123 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi057 but it was locked by another job: /home/teuthworker/archive/kchai-2022-08-27_04:01:16-rados-wip-kefu-testing-2022-08-27-0813-distro-default-smithi/6995488
2022-08-27T05:02:50.123 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi079 but it was locked by another job: /home/teuthworker/archive/kchai-2022-08-27_04:01:16-rados-wip-kefu-testing-2022-08-27-0813-distro-default-smithi/6995488