2022-03-26T06:04:00.207 INFO:root:teuthology version: 1.1.0-70390753
2022-03-26T06:04:00.371 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging
2022-03-26T06:04:00.372 INFO:teuthology.lock.ops:Updating [ubuntu@smithi038.front.sepia.ceph.com]: reset os type and version on server
2022-03-26T06:04:00.373 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2022-03-26T06:04:00.392 INFO:teuthology.lock.ops:Node 'ubuntu@smithi038.front.sepia.ceph.com' reimaging is complete
2022-03-26T06:04:00.393 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging
2022-03-26T06:04:00.393 INFO:teuthology.lock.ops:Updating [ubuntu@smithi052.front.sepia.ceph.com]: reset os type and version on server
2022-03-26T06:04:00.394 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2022-03-26T06:04:00.412 INFO:teuthology.lock.ops:Node 'ubuntu@smithi052.front.sepia.ceph.com' reimaging is complete
2022-03-26T06:04:00.413 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi178.front.sepia.ceph.com' reimaging
2022-03-26T06:04:00.413 INFO:teuthology.lock.ops:Updating [ubuntu@smithi178.front.sepia.ceph.com]: reset os type and version on server
2022-03-26T06:04:00.414 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server
2022-03-26T06:04:00.433 INFO:teuthology.lock.ops:Node 'ubuntu@smithi178.front.sepia.ceph.com' reimaging is complete
2022-03-26T06:04:00.609 INFO:teuthology.provision.fog.smithi038:Scheduling deploy of ubuntu 20.04
2022-03-26T06:04:00.637 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of ubuntu 20.04
2022-03-26T06:04:00.661 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of ubuntu 20.04
2022-03-26T06:04:00.968 INFO:teuthology.orchestra.console:Power off smithi038
2022-03-26T06:04:00.988 INFO:teuthology.orchestra.console:Power off smithi052
2022-03-26T06:04:01.026 INFO:teuthology.orchestra.console:Power off smithi178
2022-03-26T06:04:01.171 INFO:teuthology.orchestra.console:Power off for smithi178 completed
2022-03-26T06:04:01.272 INFO:teuthology.orchestra.console:Power on smithi178
2022-03-26T06:04:09.365 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2022-03-26T06:04:09.371 INFO:teuthology.orchestra.console:Power off for smithi052 completed
2022-03-26T06:04:09.467 INFO:teuthology.orchestra.console:Power on smithi038
2022-03-26T06:04:09.475 INFO:teuthology.orchestra.console:Power on smithi052
2022-03-26T06:04:09.634 INFO:teuthology.orchestra.console:Power on for smithi178 completed
2022-03-26T06:04:09.735 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish
2022-03-26T06:04:17.833 INFO:teuthology.orchestra.console:Power on for smithi038 completed
2022-03-26T06:04:17.838 INFO:teuthology.orchestra.console:Power on for smithi052 completed
2022-03-26T06:04:17.935 INFO:teuthology.provision.fog.smithi038:Waiting for deploy to finish
2022-03-26T06:04:17.939 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish
2022-03-26T06:08:55.895 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-26T06:08:55.903 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-26T06:08:56.326 INFO:teuthology.provision.fog.smithi052:Node is ready
2022-03-26T06:08:56.334 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052.front.sepia.ceph.com
2022-03-26T06:08:56.340 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52 smithi052.front.sepia.ceph.com smithi052
2022-03-26T06:08:56.445 INFO:teuthology.provision.fog.smithi038:Node is ready
2022-03-26T06:08:56.451 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038.front.sepia.ceph.com
2022-03-26T06:08:56.458 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38 smithi038.front.sepia.ceph.com smithi038
2022-03-26T06:08:56.528 INFO:teuthology.provision.fog.smithi052:Deploy complete!
2022-03-26T06:08:56.571 INFO:teuthology.provision.fog.smithi038:Deploy complete!
2022-03-26T06:09:37.588 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-03-26T06:09:38.035 INFO:teuthology.provision.fog.smithi178:Node is ready
2022-03-26T06:09:38.041 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178.front.sepia.ceph.com
2022-03-26T06:09:38.090 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178 smithi178.front.sepia.ceph.com smithi178
2022-03-26T06:09:38.290 INFO:teuthology.provision.fog.smithi178:Deploy complete!
2022-03-26T06:09:38.524 INFO:teuthology.lock.ops:Checking smithi038.front.sepia.ceph.com
2022-03-26T06:09:38.525 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-26T06:09:38.544 INFO:teuthology.lock.ops:Checking smithi052.front.sepia.ceph.com
2022-03-26T06:09:38.544 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-26T06:09:38.561 INFO:teuthology.lock.ops:Checking smithi178.front.sepia.ceph.com
2022-03-26T06:09:38.561 INFO:teuthology.lock.ops:New key found. Updating...
2022-03-26T06:09:38.578 INFO:teuthology.lock.ops:Updating [smithi038.front.sepia.ceph.com]: set os type and version on server
2022-03-26T06:09:38.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2022-03-26T06:09:38.848 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64
2022-03-26T06:09:38.855 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="Ubuntu"
2022-03-26T06:09:38.856 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-03-26T06:09:38.856 INFO:teuthology.orchestra.run.smithi038.stdout:ID=ubuntu
2022-03-26T06:09:38.856 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE=debian
2022-03-26T06:09:38.856 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-03-26T06:09:38.857 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="20.04"
2022-03-26T06:09:38.857 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://www.ubuntu.com/"
2022-03-26T06:09:38.857 INFO:teuthology.orchestra.run.smithi038.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-03-26T06:09:38.857 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-03-26T06:09:38.857 INFO:teuthology.orchestra.run.smithi038.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-03-26T06:09:38.858 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_CODENAME=focal
2022-03-26T06:09:38.858 INFO:teuthology.orchestra.run.smithi038.stdout:UBUNTU_CODENAME=focal
2022-03-26T06:09:38.859 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server
2022-03-26T06:09:38.878 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: set os type and version on server
2022-03-26T06:09:38.879 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2022-03-26T06:09:39.138 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64
2022-03-26T06:09:39.144 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="Ubuntu"
2022-03-26T06:09:39.145 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-03-26T06:09:39.145 INFO:teuthology.orchestra.run.smithi052.stdout:ID=ubuntu
2022-03-26T06:09:39.145 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE=debian
2022-03-26T06:09:39.146 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-03-26T06:09:39.146 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="20.04"
2022-03-26T06:09:39.146 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://www.ubuntu.com/"
2022-03-26T06:09:39.146 INFO:teuthology.orchestra.run.smithi052.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-03-26T06:09:39.147 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-03-26T06:09:39.147 INFO:teuthology.orchestra.run.smithi052.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-03-26T06:09:39.147 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_CODENAME=focal
2022-03-26T06:09:39.147 INFO:teuthology.orchestra.run.smithi052.stdout:UBUNTU_CODENAME=focal
2022-03-26T06:09:39.148 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2022-03-26T06:09:39.234 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: set os type and version on server
2022-03-26T06:09:39.235 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com'
2022-03-26T06:09:39.449 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64
2022-03-26T06:09:39.456 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Ubuntu"
2022-03-26T06:09:39.457 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2022-03-26T06:09:39.457 INFO:teuthology.orchestra.run.smithi178.stdout:ID=ubuntu
2022-03-26T06:09:39.457 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE=debian
2022-03-26T06:09:39.458 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2022-03-26T06:09:39.458 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="20.04"
2022-03-26T06:09:39.458 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.ubuntu.com/"
2022-03-26T06:09:39.459 INFO:teuthology.orchestra.run.smithi178.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2022-03-26T06:09:39.459 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2022-03-26T06:09:39.459 INFO:teuthology.orchestra.run.smithi178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2022-03-26T06:09:39.459 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_CODENAME=focal
2022-03-26T06:09:39.460 INFO:teuthology.orchestra.run.smithi178.stdout:UBUNTU_CODENAME=focal
2022-03-26T06:09:39.460 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server
2022-03-26T06:09:39.603 INFO:teuthology.dispatcher.supervisor:Running job 6761700
2022-03-26T06:09:39.603 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_70390753284281fb41f7552547fa9eb007fa4eef/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2022-03-25_18:42:52-rados-wip-yuri7-testing-2022-03-24-1341-pacific-distro-default-smithi/6761700 --name yuriw-2022-03-25_18:42:52-rados-wip-yuri7-testing-2022-03-24-1341-pacific-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/yuriw-2022-03-25_18:42:52-rados-wip-yuri7-testing-2022-03-24-1341-pacific-distro-default-smithi/6761700/orig.config.yaml
2022-03-26T06:09:39.606 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2022-03-25_18:42:52-rados-wip-yuri7-testing-2022-03-24-1341-pacific-distro-default-smithi/6761700
2022-03-26T06:09:39.607 INFO:teuthology.dispatcher.supervisor:Job PID: 2886
2022-03-26T06:09:39.607 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2022-03-26T12:40:08.886 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 23400s. Killing...
2022-03-26T12:40:09.432 INFO:teuthology.kill:Killing Pids: {2886}
2022-03-26T12:40:10.712 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2022-03-26T12:40:10.713 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2022-03-26T12:40:10.713 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2022-03-26T12:40:10.714 INFO:teuthology.misc:Compressing logs...
2022-03-26T12:40:10.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com'
2022-03-26T12:40:10.894 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com'
2022-03-26T12:40:11.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com'
2022-03-26T12:40:11.338 INFO:teuthology.misc:Compressing logs...
2022-03-26T12:40:12.069 INFO:teuthology.misc:Compressing logs...
2022-03-26T12:40:57.663 INFO:teuthology.kill:No teuthology processes running
2022-03-26T12:40:58.884 INFO:teuthology.kill:Nuking machines: ['smithi038', 'smithi052', 'smithi178']
2022-03-26T12:40:59.889 INFO:teuthology.kill:2022-03-26 12:40:59,889.889 INFO:teuthology.nuke:targets:
2022-03-26T12:40:59.890 INFO:teuthology.kill:  smithi038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDzafFLaYL94IOTdWpJPDOH+gGff2Tlq+9YE4mArXbKDWQlwY1m7i18sfkIR06PRZG4JWdSIvniDuS7OjLKbJGwCSOB1BE6YH353BSOc4zzLOZdtBr41ulvs8NChE4EdAwQ4Q/8iEorLjqSET19FTzRxhvND8SsrFYuOJ5nhXcOPW/KJbVJUsCUYwQXTnYC3QQF3josuBe2BZdcRFu2MouGMiHJ6vg5HiCGYWd8EK8zgLIC12xS14Om8w0e6z7Hl53j6WMH8O0NE+FR4HUgjZ94kqmxo+pA/NgzgruKIUcbaq6997izC4en6QP89UxRIVtCbUSBZQxFpDrkN3yKIugzv5R+Z5st9JXkElqgjxdZXRfD6YCkNW4J23sxSw0TcfJHN0J+QpMgVL5M7Vhq8XlWnHHG+6JAnY4WqyMQjuPvMkjv8iEnKSiiVJOlrX2t7z6N4b84tbbrN1+EoJfuBVUR+vLt7iB0LNLfQnfMqfGJVWTR2/I8YGGYFAyR6IaF4u8=
2022-03-26T12:40:59.890 INFO:teuthology.kill:  smithi052.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsYQnNglF2YXZMwwzNRbYHJk3cenAk6WV0HKkBfOAwdmSl1tvcJMGnXWxm4biM7W8Qk1UjsjiNnyE9Zp45EkGDLlwU/kMeGmanb9jFh6vStUALN/8kKCiew+qJcVWdMcafWEraagYCyy98XKGR5xCpfJ8Fmn5GnwCzwOeMjnTxqOC4fUg3CBjo9DWuZpWiq3RDI0vUUU8HhrS6JGTT+7NgfKFYwhN0LZOYu+b1kh69Br4pt7DGG9KFYGq8a841KOa2Wb9y7C2QqRL9MeXHNoTI3S3meWP1ptVMJyOnReG8ovIr32Bq29Vi5aOkOUOeVtF7XJXWOl0wd5bZuaMvOJ6VDqyxvl7Q1ZhsxpO5L84NBorNkoox0i9y4CD6cPWZCwYUfxL4+ZbqAE5IUFd8FbSNB8g387uOCuVLHtO0+DDKapzxhnB7eUwZ2geWi2SZB3AoiMTqRMJplRdMdd2lWC6KF3c2JPkTwjN/0ZiqrRaj3BDWY79lHtg8aQpYQ4G6T1k=
2022-03-26T12:40:59.891 INFO:teuthology.kill:  smithi178.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSj8CFkPnmbewUXrS2MyfLbcbojXT1AFuEKMAHnf+mQZZhF6VDXh7mXsBz0MsbsSZdpQWl4mxJuuLTPqghZ4zagVnK1YIytdhzYwYOm8WSIi64wocSV3bJFxqusKzHmPoXk8hFId6JzmXhDhksuarCupxK/OKjpfCu+KkFLqqGaWbXCWx6dJ70sbIu8vouWRXuBgKZ3MogKXpD73XZcR1YZsWJYln1F820qEMqsFAtCWQhbQgMj8J+PWpcbSzQwa8YOhFunpN7T+D0vkyVKyzUJnD96FJfCg78zWUuIBEimAOywS+t82YaX21vlT0Rl3jHxD2oSo6GpO6kjiD6x12PnNF6clLHnpO++6FT7C2tOwrpaJb1c5kbc3whcltYL80gEXJr/CLThkK4ZFYIheUZsIjbyKAxz63Pso8etMQcoHfSBxcg9JLzaZvnWE3SY+C4bdwevDMd+9nod86D3qsK0eFDjkNoljKucsnZON3xQSmwtb5s7YWzyny9TuqNCek=
2022-03-26T12:40:59.917 INFO:teuthology.kill:2022-03-26 12:40:59,917.917 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-26T12:40:59.922 INFO:teuthology.kill:2022-03-26 12:40:59,922.922 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-26T12:40:59.926 INFO:teuthology.kill:2022-03-26 12:40:59,926.926 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-03-26T12:40:59.997 INFO:teuthology.kill:2022-03-26 12:40:59,997.997 INFO:teuthology.orchestra.console:Power off smithi052
2022-03-26T12:41:00.013 INFO:teuthology.kill:2022-03-26 12:41:00,012.012 INFO:teuthology.orchestra.console:Power off smithi038
2022-03-26T12:41:00.024 INFO:teuthology.kill:2022-03-26 12:41:00,023.023 INFO:teuthology.orchestra.console:Power off smithi178
2022-03-26T12:41:12.497 INFO:teuthology.kill:2022-03-26 12:41:12,497.497 INFO:teuthology.orchestra.console:Power off for smithi052 completed
2022-03-26T12:41:12.505 INFO:teuthology.kill:2022-03-26 12:41:12,504.504 INFO:teuthology.orchestra.console:Power off for smithi038 completed
2022-03-26T12:41:12.510 INFO:teuthology.kill:2022-03-26 12:41:12,510.510 INFO:teuthology.orchestra.console:Power off for smithi178 completed
2022-03-26T12:41:12.656 INFO:teuthology.kill:2022-03-26 12:41:12,655.655 INFO:teuthology.lock.ops:unlocked: smithi178.front.sepia.ceph.com
2022-03-26T12:41:12.660 INFO:teuthology.kill:2022-03-26 12:41:12,660.660 INFO:teuthology.lock.ops:unlocked: smithi038.front.sepia.ceph.com
2022-03-26T12:41:12.661 INFO:teuthology.kill:2022-03-26 12:41:12,661.661 INFO:teuthology.lock.ops:unlocked: smithi052.front.sepia.ceph.com
2022-03-26T12:43:12.979 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2022-03-26T12:43:13.113 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi038 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-03-25_20:30:36-rgw-wip-cbodley-testing-distro-default-smithi/6762178
2022-03-26T12:43:13.113 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi052 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-03-25_20:30:36-rgw-wip-cbodley-testing-distro-default-smithi/6762178
2022-03-26T12:43:13.114 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi178 but it was locked by another job: /home/teuthworker/archive/cbodley-2022-03-25_20:30:36-rgw-wip-cbodley-testing-distro-default-smithi/6762179