2021-10-19T02:34:03.053 INFO:root:teuthology version: 1.1.0-b35344e8
2021-10-19T02:34:03.230 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi053.front.sepia.ceph.com' reimaging
2021-10-19T02:34:03.231 INFO:teuthology.lock.ops:Updating [ubuntu@smithi053.front.sepia.ceph.com]: reset os type and version on server
2021-10-19T02:34:03.231 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server
2021-10-19T02:34:03.250 INFO:teuthology.lock.ops:Node 'ubuntu@smithi053.front.sepia.ceph.com' reimaging is complete
2021-10-19T02:34:03.250 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging
2021-10-19T02:34:03.250 INFO:teuthology.lock.ops:Updating [ubuntu@smithi084.front.sepia.ceph.com]: reset os type and version on server
2021-10-19T02:34:03.251 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2021-10-19T02:34:03.272 INFO:teuthology.lock.ops:Node 'ubuntu@smithi084.front.sepia.ceph.com' reimaging is complete
2021-10-19T02:34:03.272 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi093.front.sepia.ceph.com' reimaging
2021-10-19T02:34:03.273 INFO:teuthology.lock.ops:Updating [ubuntu@smithi093.front.sepia.ceph.com]: reset os type and version on server
2021-10-19T02:34:03.273 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2021-10-19T02:34:03.293 INFO:teuthology.lock.ops:Node 'ubuntu@smithi093.front.sepia.ceph.com' reimaging is complete
2021-10-19T02:34:03.467 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of ubuntu 20.04
2021-10-19T02:34:03.569 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of ubuntu 20.04
2021-10-19T02:34:03.572 INFO:teuthology.provision.fog.smithi093:Scheduling deploy of ubuntu 20.04
2021-10-19T02:34:03.990 INFO:teuthology.orchestra.console:Power off smithi053
2021-10-19T02:34:04.039 INFO:teuthology.orchestra.console:Power off smithi084
2021-10-19T02:34:04.126 INFO:teuthology.orchestra.console:Power off smithi093
2021-10-19T02:34:12.378 INFO:teuthology.orchestra.console:Power off for smithi053 completed
2021-10-19T02:34:12.481 INFO:teuthology.orchestra.console:Power on smithi053
2021-10-19T02:34:12.492 INFO:teuthology.orchestra.console:Power off for smithi093 completed
2021-10-19T02:34:12.594 INFO:teuthology.orchestra.console:Power on smithi093
2021-10-19T02:34:16.519 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2021-10-19T02:34:16.621 INFO:teuthology.orchestra.console:Power on smithi084
2021-10-19T02:34:20.873 INFO:teuthology.orchestra.console:Power on for smithi053 completed
2021-10-19T02:34:20.880 INFO:teuthology.orchestra.console:Power on for smithi084 completed
2021-10-19T02:34:20.960 INFO:teuthology.orchestra.console:Power on for smithi093 completed
2021-10-19T02:34:20.975 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish
2021-10-19T02:34:20.982 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish
2021-10-19T02:34:21.061 INFO:teuthology.provision.fog.smithi093:Waiting for deploy to finish
2021-10-19T02:38:59.231 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-19T02:38:59.927 INFO:teuthology.provision.fog.smithi053:Node is ready
2021-10-19T02:38:59.976 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053.front.sepia.ceph.com
2021-10-19T02:38:59.989 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53 smithi053.front.sepia.ceph.com smithi053
2021-10-19T02:39:00.238 INFO:teuthology.provision.fog.smithi053:Deploy complete!
2021-10-19T02:39:21.999 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-19T02:39:22.541 INFO:teuthology.provision.fog.smithi093:Node is ready
2021-10-19T02:39:22.591 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093.front.sepia.ceph.com
2021-10-19T02:39:22.598 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93 smithi093.front.sepia.ceph.com smithi093
2021-10-19T02:39:22.788 INFO:teuthology.provision.fog.smithi093:Deploy complete!
2021-10-19T02:39:23.355 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-19T02:39:23.855 INFO:teuthology.provision.fog.smithi084:Node is ready
2021-10-19T02:39:23.862 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com
2021-10-19T02:39:23.911 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084
2021-10-19T02:39:24.122 INFO:teuthology.provision.fog.smithi084:Deploy complete!
2021-10-19T02:39:24.359 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com
2021-10-19T02:39:24.360 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-19T02:39:24.382 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com
2021-10-19T02:39:24.383 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-19T02:39:24.403 INFO:teuthology.lock.ops:Checking smithi093.front.sepia.ceph.com
2021-10-19T02:39:24.404 INFO:teuthology.lock.ops:New key found. Updating...
2021-10-19T02:39:24.426 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server
2021-10-19T02:39:24.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com'
2021-10-19T02:39:24.695 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64
2021-10-19T02:39:24.702 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Ubuntu"
2021-10-19T02:39:24.702 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-10-19T02:39:24.703 INFO:teuthology.orchestra.run.smithi053.stdout:ID=ubuntu
2021-10-19T02:39:24.703 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE=debian
2021-10-19T02:39:24.703 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-10-19T02:39:24.703 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="20.04"
2021-10-19T02:39:24.704 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.ubuntu.com/"
2021-10-19T02:39:24.704 INFO:teuthology.orchestra.run.smithi053.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-10-19T02:39:24.704 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-10-19T02:39:24.704 INFO:teuthology.orchestra.run.smithi053.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-10-19T02:39:24.704 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_CODENAME=focal
2021-10-19T02:39:24.705 INFO:teuthology.orchestra.run.smithi053.stdout:UBUNTU_CODENAME=focal
2021-10-19T02:39:24.706 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server
2021-10-19T02:39:24.726 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server
2021-10-19T02:39:24.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com'
2021-10-19T02:39:24.921 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64
2021-10-19T02:39:24.969 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Ubuntu"
2021-10-19T02:39:24.970 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-10-19T02:39:24.970 INFO:teuthology.orchestra.run.smithi084.stdout:ID=ubuntu
2021-10-19T02:39:24.971 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE=debian
2021-10-19T02:39:24.971 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-10-19T02:39:24.971 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="20.04"
2021-10-19T02:39:24.972 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.ubuntu.com/"
2021-10-19T02:39:24.974 INFO:teuthology.orchestra.run.smithi084.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-10-19T02:39:24.974 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-10-19T02:39:24.975 INFO:teuthology.orchestra.run.smithi084.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-10-19T02:39:24.975 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_CODENAME=focal
2021-10-19T02:39:24.976 INFO:teuthology.orchestra.run.smithi084.stdout:UBUNTU_CODENAME=focal
2021-10-19T02:39:24.976 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2021-10-19T02:39:24.999 INFO:teuthology.lock.ops:Updating [smithi093.front.sepia.ceph.com]: set os type and version on server
2021-10-19T02:39:24.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com'
2021-10-19T02:39:25.194 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64
2021-10-19T02:39:25.243 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Ubuntu"
2021-10-19T02:39:25.244 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="20.04.3 LTS (Focal Fossa)"
2021-10-19T02:39:25.244 INFO:teuthology.orchestra.run.smithi093.stdout:ID=ubuntu
2021-10-19T02:39:25.245 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE=debian
2021-10-19T02:39:25.245 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS"
2021-10-19T02:39:25.246 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="20.04"
2021-10-19T02:39:25.246 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.ubuntu.com/"
2021-10-19T02:39:25.247 INFO:teuthology.orchestra.run.smithi093.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-10-19T02:39:25.247 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-10-19T02:39:25.248 INFO:teuthology.orchestra.run.smithi093.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-10-19T02:39:25.248 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_CODENAME=focal
2021-10-19T02:39:25.248 INFO:teuthology.orchestra.run.smithi093.stdout:UBUNTU_CODENAME=focal
2021-10-19T02:39:25.249 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server
2021-10-19T02:39:25.351 INFO:teuthology.dispatcher.supervisor:Running job 6449645
2021-10-19T02:39:25.352 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b35344e81ac507b6cad7c1cae575ce08b9c766f2/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-10-18_19:04:28-rados-wip-yuri3-testing-2021-10-18-0907-pacific-distro-basic-smithi/6449645 --name yuriw-2021-10-18_19:04:28-rados-wip-yuri3-testing-2021-10-18-0907-pacific-distro-basic-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} -- /home/teuthworker/archive/yuriw-2021-10-18_19:04:28-rados-wip-yuri3-testing-2021-10-18-0907-pacific-distro-basic-smithi/6449645/orig.config.yaml
2021-10-19T02:39:25.357 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2021-10-18_19:04:28-rados-wip-yuri3-testing-2021-10-18-0907-pacific-distro-basic-smithi/6449645
2021-10-19T02:39:25.357 INFO:teuthology.dispatcher.supervisor:Job PID: 12316
2021-10-19T02:39:25.358 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-10-19T14:40:19.051 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-10-19T14:40:19.228 INFO:teuthology.kill:Killing Pids: {12316}
2021-10-19T14:40:20.400 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'mon.a']
2021-10-19T14:40:20.401 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['osd.4', 'osd.5', 'osd.6', 'osd.7', 'mon.b', 'mgr.x']
2021-10-19T14:40:20.401 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['osd.8', 'osd.9', 'osd.10', 'osd.11', 'mon.c']
2021-10-19T14:40:20.402 INFO:teuthology.misc:Compressing logs...
2021-10-19T14:40:20.402 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com'
2021-10-19T14:40:20.591 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com'
2021-10-19T14:40:20.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com'
2021-10-19T14:40:21.037 INFO:teuthology.misc:Compressing logs...
2021-10-19T14:40:36.305 INFO:teuthology.misc:Compressing logs...
2021-10-19T14:41:22.019 INFO:teuthology.kill:No teuthology processes running
2021-10-19T14:41:23.154 INFO:teuthology.kill:Nuking machines: ['smithi053', 'smithi084', 'smithi093']
2021-10-19T14:41:24.128 INFO:teuthology.kill:2021-10-19 14:41:24,127.127 INFO:teuthology.nuke:targets:
2021-10-19T14:41:24.129 INFO:teuthology.kill:  smithi053.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCsMJrdiC2TiP8rpBoWYGvEvyT9iSHr/60JUyT/qaV2b6lzFdeK87rgxTm8gBmuoG76JlqGuYCwutE3BvQxFeVTJSUkF/Mpipootw501CuUSBT5QdgmN8dOB4wXpCGX2gudiqsF8L+hUIOJNxrhXcsqnoRb/FohaZ1BEIGTvJs/zwpM14gX1UH/RzR4QFU6qwElzIxAxq7KyDLpHBB8AWLspwbgOr7XhZhRuifRXvDcKhQKQ4AK+MlQC+ExHnLwke7SUOPIJapjY/EwDaQOgekz4PWIzC/qRJ5RdTZ5GVzhMKpFV0s1BUli9dovfxbwkj3mAY2K3p0cnQdq62EvfQblJmBDijILVjEHGimD9wBjsfamBIYLDovhCny533C/Ympm/1EQVw9++fWr7RoyaBYt8opgoJIJW3hOIAI5mwQ/XOZZe4LYu5o5TVIPF5Oj/iLozyx+XIhSDy8ETllqb2byfVDGLQzMUTBqFF6EnXUBqB8RpdPSaYWJfyJmrXdbaNU=
2021-10-19T14:41:24.129 INFO:teuthology.kill:  smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD28gGLfTp00t2PtzHkbBxWr2MWKLbB0EyEucWueJ6EQS2PQcewkQhMWqRp+k22zGrBUpPKvrXfonEpEXx3i2pYwV0mROjF9xuaKX2/ENIXb0I4tcxZHqHI71YHjCIVMFKOHkj69h7zb1FK6HP7lTRZnMzooMXPOlPj6sbwqs9EmvMv2gCgh8weKa0WIp0u0b7LiDuYBthiGz0ZL7YSaC75nejP2K1yMgycK25wWVAFd/K2x7CGKpt6GUyyZMYNfoAfi/RSTadMvRxpQMg7MlG8o7N95mGVM9wx+NZ5CiUMeHc903o3dBmSx+XEYL1mjq2t5lEOT0RsUDv9HtuQUvsVbE4utOvjali5gMuiyAWOzgCiHwMX1WevatRaxQNSSqgbZ4xQSIhkTHtS7A05G7aYWvCW6iwU3k+OKKoXd1UBe+nDLPen5B8TTPsGiEvejibyh9oUdAhBzMAOzByzD0sstQxFmWL6w+B5895UOjacdc0jpOeCwCym/ENAVLAnlDs=
2021-10-19T14:41:24.129 INFO:teuthology.kill:  smithi093.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClYnkJpv0vT7vqjD9sCW+ZwLaQiC975SdmYeCiIOkxEQftCCpxEiQW/IC6G+id+0K4Q6nRsDKJ6cGzYS0s/rfusHG8z42rRpD9Um1S4IRttST2iOWHMDT4KBbJ5gTUTlJB1w2vmp7uG6pKMPTR22xxamq2P1gsEamOYN/3M5Hzh5LogekunngOQk02czDYbstsUwAUHnOX4IsoYvwlrke5ec1WXl4rGt8NK1YQH8SZjkVfyua6DnRqobA85Y7BcAJoLJwFyoAFY1KgGGIDXasDcX7HKxI+nwEJbLPs+q7i36bgGzwE3tdHpHsoPtczGjVK8W/lq+fV/pVAhiDc+B/yzX4mgAzm5MBBdjuTco0opsdoq3+0pS2uX/U9Zj1+oHQwd2va7wxz0xcl/rX+J7roAXCRreK507n8WpKpICczD0b8RAUlSLpqViRWaoyNTT3lkM4eUmRatroyX9zsrd4a0NTEMVN12iM+Kt/dMcC3JBZOFeKc/mPtyUbndzo4Crc=
2021-10-19T14:41:24.153 INFO:teuthology.kill:2021-10-19 14:41:24,153.153 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-19T14:41:24.157 INFO:teuthology.kill:2021-10-19 14:41:24,157.157 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-19T14:41:24.158 INFO:teuthology.kill:2021-10-19 14:41:24,158.158 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-19T14:41:24.229 INFO:teuthology.kill:2021-10-19 14:41:24,228.228 INFO:teuthology.orchestra.console:Power off smithi093
2021-10-19T14:41:24.244 INFO:teuthology.kill:2021-10-19 14:41:24,243.243 INFO:teuthology.orchestra.console:Power off smithi084
2021-10-19T14:41:24.255 INFO:teuthology.kill:2021-10-19 14:41:24,255.255 INFO:teuthology.orchestra.console:Power off smithi053
2021-10-19T14:41:36.726 INFO:teuthology.kill:2021-10-19 14:41:36,725.725 INFO:teuthology.orchestra.console:Power off for smithi093 completed
2021-10-19T14:41:36.730 INFO:teuthology.kill:2021-10-19 14:41:36,730.730 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2021-10-19T14:41:36.738 INFO:teuthology.kill:2021-10-19 14:41:36,738.738 INFO:teuthology.orchestra.console:Power off for smithi053 completed
2021-10-19T14:41:36.887 INFO:teuthology.kill:2021-10-19 14:41:36,886.886 INFO:teuthology.lock.ops:unlocked: smithi084.front.sepia.ceph.com
2021-10-19T14:41:36.890 INFO:teuthology.kill:2021-10-19 14:41:36,890.890 INFO:teuthology.lock.ops:unlocked: smithi053.front.sepia.ceph.com
2021-10-19T14:41:36.894 INFO:teuthology.kill:2021-10-19 14:41:36,894.894 INFO:teuthology.lock.ops:unlocked: smithi093.front.sepia.ceph.com
2021-10-19T14:43:37.162 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15