2022-02-02T23:04:42.366 INFO:root:teuthology version: 1.1.0-0f634dc1
2022-02-02T23:04:42.525 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi008.front.sepia.ceph.com' reimaging
2022-02-02T23:04:42.527 INFO:teuthology.lock.ops:Updating [ubuntu@smithi008.front.sepia.ceph.com]: reset os type and version on server
2022-02-02T23:04:42.527 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server
2022-02-02T23:04:42.548 INFO:teuthology.lock.ops:Node 'ubuntu@smithi008.front.sepia.ceph.com' reimaging is complete
2022-02-02T23:04:42.548 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging
2022-02-02T23:04:42.549 INFO:teuthology.lock.ops:Updating [ubuntu@smithi038.front.sepia.ceph.com]: reset os type and version on server
2022-02-02T23:04:42.549 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2022-02-02T23:04:42.568 INFO:teuthology.lock.ops:Node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging is complete
2022-02-02T23:04:42.568 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging
2022-02-02T23:04:42.569 INFO:teuthology.lock.ops:Updating [ubuntu@smithi153.front.sepia.ceph.com]: reset os type and version on server
2022-02-02T23:04:42.569 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2022-02-02T23:04:42.592 INFO:teuthology.lock.ops:Node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging is complete
2022-02-02T23:04:42.770 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of ubuntu 20.04
2022-02-02T23:04:42.794 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of ubuntu 20.04
2022-02-02T23:04:42.796 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of ubuntu 20.04
2022-02-02T23:04:43.139 INFO:teuthology.orchestra.console:Power off smithi153
2022-02-02T23:04:43.155 INFO:teuthology.orchestra.console:Power off smithi008
2022-02-02T23:04:43.166 INFO:teuthology.orchestra.console:Power off smithi038
2022-02-02T23:04:51.532 INFO:teuthology.orchestra.console:Power off for smithi008 completed
2022-02-02T23:04:51.635 INFO:teuthology.orchestra.console:Power on smithi008
2022-02-02T23:04:55.638 INFO:teuthology.orchestra.console:Power off for smithi153 completed
2022-02-02T23:04:55.650 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2022-02-02T23:04:55.740 INFO:teuthology.orchestra.console:Power on smithi153
2022-02-02T23:04:55.752 INFO:teuthology.orchestra.console:Power on smithi038
2022-02-02T23:05:00.005 INFO:teuthology.orchestra.console:Power on for smithi008 completed
2022-02-02T23:05:00.106 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish
2022-02-02T23:05:04.110 INFO:teuthology.orchestra.console:Power on for smithi153 completed
2022-02-02T23:05:04.121 INFO:teuthology.orchestra.console:Power on for smithi038 completed
2022-02-02T23:05:04.212 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish
2022-02-02T23:05:04.223 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2022-02-02T23:09:47.414 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-02T23:09:47.943 INFO:teuthology.provision.fog.smithi008:Node is ready
2022-02-02T23:09:47.950 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com
2022-02-02T23:09:47.957 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008
2022-02-02T23:09:48.108 INFO:teuthology.provision.fog.smithi008:Deploy complete!
2022-02-02T23:09:51.448 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-02T23:09:51.999 INFO:teuthology.provision.fog.smithi038:Node is ready
2022-02-02T23:09:52.006 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2022-02-02T23:09:52.055 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2022-02-02T23:09:52.214 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2022-02-02T23:10:11.185 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-02-02T23:10:11.799 INFO:teuthology.provision.fog.smithi153:Node is ready
2022-02-02T23:10:11.806 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com
2022-02-02T23:10:11.855 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153
2022-02-02T23:10:12.012 INFO:teuthology.provision.fog.smithi153:Deploy complete!
2022-02-02T23:10:12.235 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com
2022-02-02T23:10:12.236 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-02T23:10:12.255 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com
2022-02-02T23:10:12.255 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-02T23:10:12.273 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com
2022-02-02T23:10:12.273 INFO:teuthology.lock.ops:New key found. Updating...
2022-02-02T23:10:12.290 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server
2022-02-02T23:10:12.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2022-02-02T23:10:12.609 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64
2022-02-02T23:10:12.616 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="Ubuntu"
2022-02-02T23:10:12.616 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-02T23:10:12.617 INFO:teuthology.orchestra.run.smithi008.stdout:ID=ubuntu
2022-02-02T23:10:12.617 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE=debian
2022-02-02T23:10:12.617 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-02T23:10:12.617 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="20.04"
2022-02-02T23:10:12.618 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-02T23:10:12.618 INFO:teuthology.orchestra.run.smithi008.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-02T23:10:12.618 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-02T23:10:12.618 INFO:teuthology.orchestra.run.smithi008.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-02T23:10:12.619 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_CODENAME=focal
2022-02-02T23:10:12.619 INFO:teuthology.orchestra.run.smithi008.stdout:UBUNTU_CODENAME=focal
2022-02-02T23:10:12.620 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server
2022-02-02T23:10:12.638 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server
2022-02-02T23:10:12.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2022-02-02T23:10:12.910 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64
2022-02-02T23:10:12.916 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Ubuntu"
2022-02-02T23:10:12.916 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-02T23:10:12.916 INFO:teuthology.orchestra.run.smithi038.stdout:ID=ubuntu
2022-02-02T23:10:12.917 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE=debian
2022-02-02T23:10:12.917 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-02T23:10:12.917 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="20.04"
2022-02-02T23:10:12.917 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-02T23:10:12.917 INFO:teuthology.orchestra.run.smithi038.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-02T23:10:12.918 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-02T23:10:12.918 INFO:teuthology.orchestra.run.smithi038.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-02T23:10:12.918 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_CODENAME=focal
2022-02-02T23:10:12.918 INFO:teuthology.orchestra.run.smithi038.stdout:UBUNTU_CODENAME=focal
2022-02-02T23:10:12.919 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2022-02-02T23:10:12.936 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server
2022-02-02T23:10:12.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2022-02-02T23:10:13.115 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64
2022-02-02T23:10:13.121 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Ubuntu"
2022-02-02T23:10:13.122 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-02-02T23:10:13.122 INFO:teuthology.orchestra.run.smithi153.stdout:ID=ubuntu
2022-02-02T23:10:13.122 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE=debian
2022-02-02T23:10:13.122 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-02-02T23:10:13.123 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="20.04"
2022-02-02T23:10:13.123 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.ubuntu.com/"
2022-02-02T23:10:13.123 INFO:teuthology.orchestra.run.smithi153.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-02-02T23:10:13.124 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-02-02T23:10:13.124 INFO:teuthology.orchestra.run.smithi153.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-02-02T23:10:13.124 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_CODENAME=focal
2022-02-02T23:10:13.124 INFO:teuthology.orchestra.run.smithi153.stdout:UBUNTU_CODENAME=focal
2022-02-02T23:10:13.125 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server
2022-02-02T23:10:13.225 INFO:teuthology.dispatcher.supervisor:Running job 6656016
2022-02-02T23:10:13.225 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_0f634dc1c0a64a93646e6d007882c69b3e1663e0/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-02-02_19:24:07-rados-wip-yuri5-testing-2022-02-02-0936-pacific-distro-default-smithi/6656016 --name yuriw-2022-02-02_19:24:07-rados-wip-yuri5-testing-2022-02-02-0936-pacific-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/yuriw-2022-02-02_19:24:07-rados-wip-yuri5-testing-2022-02-02-0936-pacific-distro-default-smithi/6656016/orig.config.yaml
2022-02-02T23:10:13.228 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-02-02_19:24:07-rados-wip-yuri5-testing-2022-02-02-0936-pacific-distro-default-smithi/6656016
2022-02-02T23:10:13.229 INFO:teuthology.dispatcher.supervisor:Job PID: 22478
2022-02-02T23:10:13.229 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-02-03T05:40:43.491 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-02-03T05:40:43.680 INFO:teuthology.kill:Killing Pids: {22478}
2022-02-03T05:40:44.910 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-02-03T05:40:44.911 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-02-03T05:40:44.912 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-02-03T05:40:44.912 INFO:teuthology.misc:Compressing logs...
2022-02-03T05:40:44.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com'
2022-02-03T05:40:45.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2022-02-03T05:40:45.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com'
2022-02-03T05:40:45.646 INFO:teuthology.misc:Compressing logs...
2022-02-03T05:40:46.375 INFO:teuthology.misc:Compressing logs...
2022-02-03T05:41:24.424 INFO:teuthology.kill:No teuthology processes running
2022-02-03T05:41:25.583 INFO:teuthology.kill:Nuking machines: ['smithi008', 'smithi038', 'smithi153']
2022-02-03T05:41:26.579 INFO:teuthology.kill:2022-02-03 05:41:26,578.578 INFO:teuthology.nuke:targets:
2022-02-03T05:41:26.580 INFO:teuthology.kill:  smithi008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZ64Rok8kqv6KW7KN936IXU1B0EMiueqzU3rY3dAEMYyFIbRuNRzawya6yUC4n7s2t7x9L/nBxJKqrl3NoKW2j4NUPhcJzM6d0W2jcFYhOqcfiPOqRiEUUSSg2bi9GnZKCmkAirfxegDClQXW/FxmJsRLGocc/k/foXiVK0zpECGcmzw06yYX8RT0vrHr20fuQfSsxsRscrmLQFr8tIKILjroJr4MNDuWqCkBt8FYUfucj9s16kIHXZT6evmNz5eh0d5vW3vXAzwIvdCy7sQk+onZkpDFRKRffBhOi9SashyoFRTlyYUv2cCIXapHNFHQRH0XkoIoK8v1fdg9YM0ZI+/khgbiMN3fJHlf71GBLdU7krgKWyU0PzN+n7LS16kJQApiuQQJdzCADLZJhbGjgdmRq/DAVXTgY526+JnlxhRYP9DRKEakg07nGSI8IJl9YSB/djOvtAN9WURQQGTwYS0ReaGTenehb01yofg1vOZxc9gU9JQs/+t/RyXbAMM=
2022-02-03T05:41:26.580 INFO:teuthology.kill:  smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDqaulmG8Qni3i5Z7vipHI5d1ms8D/kn7XWpZbDfHv4ocjYP5sc/ayWqLWxsZMliL/Czvd2n701k2NY4RoCrfhkFwyhjFexkvF+mZ90Ra7xUTyVAjIDKt/WfRWJusokT5wZDXCnC1sCuy8m0t8kpt3dOMW0qF0qc2OhlUHL2bdfeFxv1C0omoDhfDIgRLX3d7iQ72taAFfFKoqNmjtkpIqCDrLirtQ+gUP+LB/RG2fQW/QDUiSz9Q78dmYgcjs3CKqwC2V35668OiQ4EVk5ysOhHfssGBTAh3ITVrrsmE8bp5M+LOfrEAOTgILkgJKEI2sYeu4z6dfp1+ek9OrZSQAYDYTCVOUvvVfTbEmNVjUiXXMDEeWPmBzvugWYaHDFsX4banJN7SJ5YY+SMOZP/tXiaKr3hpFqW5pnG/XhsadDE2HkwkctP8Ntbe9TDrm+HZzi29TwYs3RFjNWVKuMaCqpCUItRJH2pqbdKiplXV6e9+eQASl0Q/vyFgLG66gURhs=
2022-02-03T05:41:26.581 INFO:teuthology.kill:  smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/3hknz5s1/y+OjW72loIdKv+HbgfB3qlfYCnWrVslrorigahNogUDRsSD35tvOfTdBmmFMPNmoTpB4jRsguAzJDvo9UIWSBJW5lFBrsIWNoWc01FIq0e389kMTfGGJqaDpKJD/um//2uIGT7PlIHf5oYSgPMi21PVDNsTEsshHvKTLsnzdVC1/9p3RM3Q8dSv3WZMPWOZ/+MpLaS7afdH557d7jhYMvH5e968/APkvCBtmuIefhjInoQ9qzimYgj6+XmCYnlFf2NJHmaT1LxigcgdIeeBnyzryDsxejsf46CRAmIPiAqSbhzCxHXZNlPOauh/+r6ySLfc8DeHvR9sMAFCq0djJ8w8aTl5amO4bQKLFkz9n4mzHlBzUq+OfzRyECLypkBa3TS24JERslnyayZwwRa+aHpHlEL2atMeTdRlJYBW1XAbvFA8GLM/4mnP7cxwZe/0yzBae0tPZkuvjTl34/5ffAsFuUiIFAg3IssxQ+QiUmTguNMkbx5MArk=
2022-02-03T05:41:26.606 INFO:teuthology.kill:2022-02-03 05:41:26,606.606 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-03T05:41:26.610 INFO:teuthology.kill:2022-02-03 05:41:26,610.610 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-03T05:41:26.611 INFO:teuthology.kill:2022-02-03 05:41:26,611.611 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-02-03T05:41:26.691 INFO:teuthology.kill:2022-02-03 05:41:26,691.691 INFO:teuthology.orchestra.console:Power off smithi008
2022-02-03T05:41:26.705 INFO:teuthology.kill:2022-02-03 05:41:26,705.705 INFO:teuthology.orchestra.console:Power off smithi153
2022-02-03T05:41:26.716 INFO:teuthology.kill:2022-02-03 05:41:26,715.715 INFO:teuthology.orchestra.console:Power off smithi038
2022-02-03T05:41:39.189 INFO:teuthology.kill:2022-02-03 05:41:39,189.189 INFO:teuthology.orchestra.console:Power off for smithi008 completed
2022-02-03T05:41:39.195 INFO:teuthology.kill:2022-02-03 05:41:39,195.195 INFO:teuthology.orchestra.console:Power off for smithi153 completed
2022-02-03T05:41:39.201 INFO:teuthology.kill:2022-02-03 05:41:39,201.201 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2022-02-03T05:41:39.337 INFO:teuthology.kill:2022-02-03 05:41:39,337.337 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com
2022-02-03T05:41:39.340 INFO:teuthology.kill:2022-02-03 05:41:39,340.340 INFO:teuthology.lock.ops:unlocked: smithi008.front.sepia.ceph.com
2022-02-03T05:41:39.347 INFO:teuthology.kill:2022-02-03 05:41:39,347.347 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com
2022-02-03T05:43:39.679 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-02-03T05:43:39.801 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi008 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-03_02:33:02-rados-wip-yuri8-testing-2022-02-02-1557-quincy-distro-default-smithi/6657379
2022-02-03T05:43:39.801 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi038 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-03_02:33:02-rados-wip-yuri8-testing-2022-02-02-1557-quincy-distro-default-smithi/6657378
2022-02-03T05:43:39.802 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/yuriw-2022-02-03_02:33:02-rados-wip-yuri8-testing-2022-02-02-1557-quincy-distro-default-smithi/6657380