2023-01-27T20:50:53.289 INFO:root:teuthology version: 0.0.1.dev31+g912c45c 2023-01-27T20:50:53.442 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging 2023-01-27T20:50:53.443 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server 2023-01-27T20:50:53.443 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-01-27T20:50:53.459 INFO:teuthology.lock.ops:Node 'smithi016.front.sepia.ceph.com' reimaging is complete 2023-01-27T20:50:53.460 INFO:teuthology.lock.ops:Start node 'smithi137.front.sepia.ceph.com' reimaging 2023-01-27T20:50:53.460 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: reset os type and version on server 2023-01-27T20:50:53.461 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2023-01-27T20:50:53.478 INFO:teuthology.lock.ops:Node 'smithi137.front.sepia.ceph.com' reimaging is complete 2023-01-27T20:50:53.479 INFO:teuthology.lock.ops:Start node 'smithi148.front.sepia.ceph.com' reimaging 2023-01-27T20:50:53.479 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: reset os type and version on server 2023-01-27T20:50:53.479 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-01-27T20:50:53.496 INFO:teuthology.lock.ops:Node 'smithi148.front.sepia.ceph.com' reimaging is complete 2023-01-27T20:50:53.595 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of ubuntu 22.04 2023-01-27T20:50:53.639 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of ubuntu 22.04 2023-01-27T20:50:53.641 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of ubuntu 22.04 2023-01-27T20:50:53.803 INFO:teuthology.orchestra.console:Power off smithi016 2023-01-27T20:50:53.843 INFO:teuthology.orchestra.console:Power off smithi148 2023-01-27T20:50:53.857 INFO:teuthology.orchestra.console:Power off smithi137 2023-01-27T20:50:53.953 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2023-01-27T20:50:53.985 INFO:teuthology.orchestra.console:Power off for smithi148 completed 2023-01-27T20:50:53.999 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2023-01-27T20:50:54.054 INFO:teuthology.orchestra.console:Power on smithi016 2023-01-27T20:50:54.087 INFO:teuthology.orchestra.console:Power on smithi148 2023-01-27T20:50:54.100 INFO:teuthology.orchestra.console:Power on smithi137 2023-01-27T20:50:58.337 INFO:teuthology.orchestra.console:Power on for smithi148 completed 2023-01-27T20:50:58.342 INFO:teuthology.orchestra.console:Power on for smithi016 completed 2023-01-27T20:50:58.439 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish 2023-01-27T20:50:58.444 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2023-01-27T20:51:02.459 INFO:teuthology.orchestra.console:Power on for smithi137 completed 2023-01-27T20:51:02.560 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish 2023-01-27T20:54:03.688 ERROR:teuthology.orchestra.connection:Error authenticating with smithi137.front.sepia.ceph.com: Authentication failed. 2023-01-27T20:54:17.708 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-01-27T20:54:26.764 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-01-27T20:54:32.809 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-01-27T20:54:35.816 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-01-27T20:54:41.864 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-01-27T20:54:50.920 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-01-27T20:54:59.980 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-01-27T20:55:04.692 WARNING:teuthology.provision.fog:timed out 2023-01-27T20:55:09.036 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-01-27T20:55:29.128 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-01-27T20:55:38.184 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-01-27T20:55:41.819 WARNING:teuthology.provision.fog:timed out 2023-01-27T20:55:44.187 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-01-27T20:55:48.080 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-27T20:55:48.537 INFO:teuthology.provision.fog.smithi016:Node is ready 2023-01-27T20:55:48.586 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com 2023-01-27T20:55:48.635 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016 2023-01-27T20:55:48.945 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2023-01-27T20:55:50.189 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-01-27T20:55:56.192 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-01-27T20:56:02.194 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-01-27T20:56:08.404 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-27T20:56:08.846 INFO:teuthology.provision.fog.smithi137:Node is ready 2023-01-27T20:56:08.891 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com 2023-01-27T20:56:08.939 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137 2023-01-27T20:56:09.262 INFO:teuthology.provision.fog.smithi137:Deploy complete! 2023-01-27T20:56:15.040 WARNING:teuthology.provision.fog:timed out 2023-01-27T20:56:21.353 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-27T20:56:21.748 INFO:teuthology.provision.fog.smithi148:Node is ready 2023-01-27T20:56:21.796 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148.front.sepia.ceph.com 2023-01-27T20:56:21.844 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148 2023-01-27T20:56:22.138 INFO:teuthology.provision.fog.smithi148:Deploy complete! 2023-01-27T20:56:22.394 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2023-01-27T20:56:22.394 INFO:teuthology.lock.ops:New key found. Updating... 2023-01-27T20:56:22.417 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com 2023-01-27T20:56:22.417 INFO:teuthology.lock.ops:New key found. Updating... 2023-01-27T20:56:22.432 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com 2023-01-27T20:56:22.433 INFO:teuthology.lock.ops:New key found. Updating... 2023-01-27T20:56:22.449 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2023-01-27T20:56:22.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-01-27T20:56:22.821 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-01-27T20:56:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-01-27T20:56:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Ubuntu" 2023-01-27T20:56:22.870 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="22.04" 2023-01-27T20:56:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-01-27T20:56:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_CODENAME=jammy 2023-01-27T20:56:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:ID=ubuntu 2023-01-27T20:56:22.871 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE=debian 2023-01-27T20:56:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.ubuntu.com/" 2023-01-27T20:56:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-01-27T20:56:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-01-27T20:56:22.872 INFO:teuthology.orchestra.run.smithi016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-01-27T20:56:22.873 INFO:teuthology.orchestra.run.smithi016.stdout:UBUNTU_CODENAME=jammy 2023-01-27T20:56:22.874 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-01-27T20:56:22.890 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server 2023-01-27T20:56:22.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2023-01-27T20:56:23.131 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2023-01-27T20:56:23.179 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-01-27T20:56:23.179 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Ubuntu" 2023-01-27T20:56:23.179 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="22.04" 2023-01-27T20:56:23.179 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-01-27T20:56:23.180 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_CODENAME=jammy 2023-01-27T20:56:23.180 INFO:teuthology.orchestra.run.smithi137.stdout:ID=ubuntu 2023-01-27T20:56:23.180 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE=debian 2023-01-27T20:56:23.180 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.ubuntu.com/" 2023-01-27T20:56:23.180 INFO:teuthology.orchestra.run.smithi137.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-01-27T20:56:23.181 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-01-27T20:56:23.181 INFO:teuthology.orchestra.run.smithi137.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-01-27T20:56:23.181 INFO:teuthology.orchestra.run.smithi137.stdout:UBUNTU_CODENAME=jammy 2023-01-27T20:56:23.181 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2023-01-27T20:56:23.195 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server 2023-01-27T20:56:23.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-01-27T20:56:23.411 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2023-01-27T20:56:23.459 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-01-27T20:56:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Ubuntu" 2023-01-27T20:56:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="22.04" 2023-01-27T20:56:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-01-27T20:56:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_CODENAME=jammy 2023-01-27T20:56:23.460 INFO:teuthology.orchestra.run.smithi148.stdout:ID=ubuntu 2023-01-27T20:56:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE=debian 2023-01-27T20:56:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.ubuntu.com/" 2023-01-27T20:56:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-01-27T20:56:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-01-27T20:56:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-01-27T20:56:23.461 INFO:teuthology.orchestra.run.smithi148.stdout:UBUNTU_CODENAME=jammy 2023-01-27T20:56:23.462 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-01-27T20:56:23.554 INFO:teuthology.dispatcher.supervisor:Running job 7140401 2023-01-27T20:56:23.555 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_912c45c320f1f67e29963a9ee0910819e4e6e16c/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-01-27_01:12:43-krbd-wip-yuri6-testing-2023-01-26-0941-testing-default-smithi/7140401 --name yuriw-2023-01-27_01:12:43-krbd-wip-yuri6-testing-2023-01-26-0941-testing-default-smithi --description krbd/thrash/{bluestore-bitmap ceph/ceph clusters/fixed-3 conf ms_mode$/{legacy} thrashers/backoff thrashosds-health workloads/rbd_workunit_suites_ffsb} -- /home/teuthworker/archive/yuriw-2023-01-27_01:12:43-krbd-wip-yuri6-testing-2023-01-26-0941-testing-default-smithi/7140401/orig.config.yaml 2023-01-27T20:56:23.558 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-01-27_01:12:43-krbd-wip-yuri6-testing-2023-01-26-0941-testing-default-smithi/7140401 2023-01-27T20:56:23.559 INFO:teuthology.dispatcher.supervisor:Job PID: 6725 2023-01-27T20:56:23.559 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-01-28T08:56:43.154 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2023-01-28T08:56:43.310 INFO:teuthology.kill:Killing Pids: {6725} 2023-01-28T08:56:44.210 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2023-01-28T08:56:44.211 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-01-28T08:56:44.211 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['client.0'] 2023-01-28T08:56:44.211 INFO:teuthology.misc:Compressing logs... 2023-01-28T08:56:44.212 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-01-28T08:56:44.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2023-01-28T08:56:44.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-01-28T08:56:45.141 INFO:teuthology.misc:Compressing logs... 2023-01-28T08:56:46.444 INFO:teuthology.misc:Compressing logs... 2023-01-28T08:56:47.980 INFO:teuthology.kill:No teuthology processes running 2023-01-28T08:56:48.766 INFO:teuthology.kill:Nuking machines: ['smithi016', 'smithi137', 'smithi148'] 2023-01-28T08:56:49.420 INFO:teuthology.kill:2023-01-28 08:56:49,420.420 INFO:teuthology.nuke:targets: 2023-01-28T08:56:49.421 INFO:teuthology.kill: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBISP8/MPoiRAwHBMQo/eiocX1IOxnIMktHGdHEpArrnYWAAHFy8n1wkVnXUHc63z/k/QDod2AYfIfqLdhiI2CiE= 2023-01-28T08:56:49.421 INFO:teuthology.kill: smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNMBTncfnry4DJGyLGtxUV1h/TvzeHWOf9ipA1BGD2Q+0XDA+nkBVhyh/B6u6/Qp/YvObhZFcvW8Lrk4gpaV7Cg= 2023-01-28T08:56:49.422 INFO:teuthology.kill: smithi148.front.sepia.ceph.com: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHvWUJRxlJHz1h1W57v/yiCcQpNvM2rGuGVUyc9bxpEd 2023-01-28T08:56:49.422 INFO:teuthology.kill:2023-01-28 08:56:49,420.420 INFO:teuthology.nuke:Checking targets against current locks 2023-01-28T08:56:49.473 INFO:teuthology.kill:2023-01-28 08:56:49,472.472 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-28T08:56:49.502 INFO:teuthology.kill:2023-01-28 08:56:49,502.502 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-28T08:56:49.530 INFO:teuthology.kill:2023-01-28 08:56:49,530.530 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-28T08:56:49.567 INFO:teuthology.kill:2023-01-28 08:56:49,566.566 INFO:teuthology.orchestra.console:Power off smithi016 2023-01-28T08:56:49.587 INFO:teuthology.kill:2023-01-28 08:56:49,587.587 INFO:teuthology.orchestra.console:Power off smithi137 2023-01-28T08:56:49.632 INFO:teuthology.kill:2023-01-28 08:56:49,631.631 INFO:teuthology.orchestra.console:Power off smithi148 2023-01-28T08:56:57.993 INFO:teuthology.kill:2023-01-28 08:56:57,992.992 INFO:teuthology.orchestra.console:Power off for smithi148 completed 2023-01-28T08:56:58.143 INFO:teuthology.kill:2023-01-28 08:56:58,143.143 INFO:teuthology.lock.ops:unlocked: smithi148.front.sepia.ceph.com 2023-01-28T08:57:02.054 INFO:teuthology.kill:2023-01-28 08:57:02,054.054 INFO:teuthology.orchestra.console:Power off for smithi016 completed 2023-01-28T08:57:02.060 INFO:teuthology.kill:2023-01-28 08:57:02,060.060 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2023-01-28T08:57:02.208 INFO:teuthology.kill:2023-01-28 08:57:02,207.207 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com 2023-01-28T08:57:02.209 INFO:teuthology.kill:2023-01-28 08:57:02,209.209 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com 2023-01-28T08:59:02.428 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2023-01-28T08:59:02.540 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi016 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-27_01:26:39-powercycle-wip-yuri6-testing-2023-01-26-0941-distro-default-smithi/7141154 2023-01-28T08:59:02.540 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi137 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-27_01:26:39-powercycle-wip-yuri6-testing-2023-01-26-0941-distro-default-smithi/7141154 2023-01-28T08:59:02.540 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi148 but it was locked by another job: /home/teuthworker/archive/yuriw-2023-01-27_01:26:39-powercycle-wip-yuri6-testing-2023-01-26-0941-distro-default-smithi/7141154