2021-06-05T17:33:14.453 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-05T17:33:14.563 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi031.front.sepia.ceph.com' reimaging 2021-06-05T17:33:14.564 INFO:teuthology.lock.ops:Updating [ubuntu@smithi031.front.sepia.ceph.com]: reset os type and version on server 2021-06-05T17:33:14.565 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2021-06-05T17:33:14.579 INFO:teuthology.lock.ops:Node 'ubuntu@smithi031.front.sepia.ceph.com' reimaging is complete 2021-06-05T17:33:14.580 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi097.front.sepia.ceph.com' reimaging 2021-06-05T17:33:14.580 INFO:teuthology.lock.ops:Updating [ubuntu@smithi097.front.sepia.ceph.com]: reset os type and version on server 2021-06-05T17:33:14.580 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2021-06-05T17:33:14.596 INFO:teuthology.lock.ops:Node 'ubuntu@smithi097.front.sepia.ceph.com' reimaging is complete 2021-06-05T17:33:14.597 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging 2021-06-05T17:33:14.597 INFO:teuthology.lock.ops:Updating [ubuntu@smithi153.front.sepia.ceph.com]: reset os type and version on server 2021-06-05T17:33:14.597 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2021-06-05T17:33:14.613 INFO:teuthology.lock.ops:Node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging is complete 2021-06-05T17:33:14.781 INFO:teuthology.provision.fog.smithi031:Scheduling deploy of rhel 8.4 2021-06-05T17:33:14.795 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of rhel 8.4 2021-06-05T17:33:14.807 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of rhel 8.4 2021-06-05T17:33:15.122 INFO:teuthology.orchestra.console:Power off smithi097 2021-06-05T17:33:15.155 INFO:teuthology.orchestra.console:Power off smithi031 2021-06-05T17:33:15.169 INFO:teuthology.orchestra.console:Power off smithi153 2021-06-05T17:33:15.276 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2021-06-05T17:33:15.297 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2021-06-05T17:33:15.310 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-06-05T17:33:15.378 INFO:teuthology.orchestra.console:Power on smithi097 2021-06-05T17:33:15.403 INFO:teuthology.orchestra.console:Power on smithi031 2021-06-05T17:33:15.411 INFO:teuthology.orchestra.console:Power on smithi153 2021-06-05T17:33:23.743 INFO:teuthology.orchestra.console:Power on for smithi097 completed 2021-06-05T17:33:23.762 INFO:teuthology.orchestra.console:Power on for smithi031 completed 2021-06-05T17:33:23.768 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2021-06-05T17:33:23.845 INFO:teuthology.provision.fog.smithi097:Waiting for deploy to finish 2021-06-05T17:33:23.864 INFO:teuthology.provision.fog.smithi031:Waiting for deploy to finish 2021-06-05T17:33:23.869 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2021-06-05T17:37:32.021 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-05T17:37:32.226 INFO:teuthology.provision.fog.smithi031:Node is ready 2021-06-05T17:37:32.243 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031.front.sepia.ceph.com 2021-06-05T17:37:32.299 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31 smithi031.front.sepia.ceph.com smithi031 2021-06-05T17:37:32.632 INFO:teuthology.provision.fog.smithi031:Deploy complete! 2021-06-05T17:37:32.648 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-05T17:37:41.586 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-05T17:37:53.004 INFO:teuthology.provision.fog.smithi097:Node is ready 2021-06-05T17:37:53.062 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097.front.sepia.ceph.com 2021-06-05T17:37:53.119 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97 smithi097.front.sepia.ceph.com smithi097 2021-06-05T17:37:53.469 INFO:teuthology.provision.fog.smithi097:Deploy complete! 2021-06-05T17:37:56.865 INFO:teuthology.provision.fog.smithi153:Node is ready 2021-06-05T17:37:56.924 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2021-06-05T17:37:56.981 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2021-06-05T17:37:57.306 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2021-06-05T17:37:57.444 INFO:teuthology.lock.ops:Checking smithi031.front.sepia.ceph.com 2021-06-05T17:37:57.445 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-05T17:37:57.462 INFO:teuthology.lock.ops:Checking smithi097.front.sepia.ceph.com 2021-06-05T17:37:57.463 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-05T17:37:57.474 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2021-06-05T17:37:57.475 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-05T17:37:57.488 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: set os type and version on server 2021-06-05T17:37:57.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-05T17:37:57.725 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2021-06-05T17:37:57.781 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Red Hat Enterprise Linux" 2021-06-05T17:37:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8.4 (Ootpa)" 2021-06-05T17:37:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:ID="rhel" 2021-06-05T17:37:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="fedora" 2021-06-05T17:37:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8.4" 2021-06-05T17:37:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2021-06-05T17:37:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-06-05T17:37:57.783 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2021-06-05T17:37:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-06-05T17:37:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.redhat.com/" 2021-06-05T17:37:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-06-05T17:37:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-06-05T17:37:57.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2021-06-05T17:37:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-06-05T17:37:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-06-05T17:37:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-06-05T17:37:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-06-05T17:37:57.787 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2021-06-05T17:37:57.802 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: set os type and version on server 2021-06-05T17:37:57.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-05T17:37:58.036 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2021-06-05T17:37:58.091 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2021-06-05T17:37:58.092 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.4 (Ootpa)" 2021-06-05T17:37:58.092 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2021-06-05T17:37:58.092 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2021-06-05T17:37:58.092 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.4" 2021-06-05T17:37:58.093 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2021-06-05T17:37:58.093 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-06-05T17:37:58.093 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2021-06-05T17:37:58.093 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-06-05T17:37:58.094 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2021-06-05T17:37:58.094 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-06-05T17:37:58.094 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-06-05T17:37:58.095 INFO:teuthology.orchestra.run.smithi097.stdout: 2021-06-05T17:37:58.095 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-06-05T17:37:58.095 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-06-05T17:37:58.095 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-06-05T17:37:58.096 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-06-05T17:37:58.096 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2021-06-05T17:37:58.111 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2021-06-05T17:37:58.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-05T17:37:58.299 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2021-06-05T17:37:58.355 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Red Hat Enterprise Linux" 2021-06-05T17:37:58.355 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8.4 (Ootpa)" 2021-06-05T17:37:58.355 INFO:teuthology.orchestra.run.smithi153.stdout:ID="rhel" 2021-06-05T17:37:58.356 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="fedora" 2021-06-05T17:37:58.356 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8.4" 2021-06-05T17:37:58.356 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2021-06-05T17:37:58.356 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2021-06-05T17:37:58.357 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2021-06-05T17:37:58.357 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2021-06-05T17:37:58.357 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.redhat.com/" 2021-06-05T17:37:58.358 INFO:teuthology.orchestra.run.smithi153.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2021-06-05T17:37:58.358 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-06-05T17:37:58.358 INFO:teuthology.orchestra.run.smithi153.stdout: 2021-06-05T17:37:58.358 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-06-05T17:37:58.359 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2021-06-05T17:37:58.359 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-06-05T17:37:58.359 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2021-06-05T17:37:58.360 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2021-06-05T17:37:58.436 INFO:teuthology.dispatcher.supervisor:Running job 6154566 2021-06-05T17:37:58.437 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2021-06-05_16:50:09-fs-wip-pdonnell-testing-20210605.021627-distro-basic-smithi/6154566 --name pdonnell-2021-06-05_16:50:09-fs-wip-pdonnell-testing-20210605.021627-distro-basic-smithi --description fs/workload/{begin clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{frag_enable osd-asserts session_timeout whitelist_health whitelist_wrongly_marked_down} ranks/5 scrub/no standby-replay tasks/{0-check-counter workunit/suites/blogbench} wsync/{no}} -- /home/teuthworker/archive/pdonnell-2021-06-05_16:50:09-fs-wip-pdonnell-testing-20210605.021627-distro-basic-smithi/6154566/orig.config.yaml 2021-06-05T17:37:58.440 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2021-06-05_16:50:09-fs-wip-pdonnell-testing-20210605.021627-distro-basic-smithi/6154566 2021-06-05T17:37:58.441 INFO:teuthology.dispatcher.supervisor:Job PID: 29493 2021-06-05T17:37:58.441 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-06T05:38:48.603 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-06T05:38:48.809 INFO:teuthology.kill:Killing Pids: {29493} 2021-06-06T05:38:49.919 INFO:teuthology.kill:Nuking machines: ['smithi031', 'smithi097', 'smithi153'] 2021-06-06T05:38:50.848 INFO:teuthology.kill:2021-06-06 05:38:50,848.848 INFO:teuthology.nuke:targets: 2021-06-06T05:38:50.849 INFO:teuthology.kill: smithi031.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMUUOdMqODXQ+7gEj5Wn+eMJWOdn1iI4D3X3ZYWS9p/MTw/VaXGlkHvAdBfAeNrYHqkhI1MBN95pVoA7W+fF2SnO/RyeI4HKRVNMrf/qsc8pTpi/crpmRIkfUV7bBxvCisZfMwJbu3KuBsOFhcUonlAXqNB+3Zcm/rXIcaI9Ezzc9+3ObfEDwQVUf75dWEgtw49Jz8+6gvEzEFMz/GyHRVIKxbrjpF5fOG2mF4szQTDWSxh6kmmHqr+K2jGfJJjUpjZVB4Ycwhn9M0EaUKIlc4bK+2UQqhe9uqTmWhiCG48wX0HUq/SBb6ps5wJpuQ02Nd+MjG/KlZJ77MHfCUrD69k8gIjnzj/49RTecmABcEKS6eBX2sJfvXgNFBEuxYKlEPx1t2uKf+4LxCjB8ooZLN6vgjTcdd9Eb5hRvL0PED2xfnQZyGrUgFiUyab4EvkNNkuk4hkG6/HS6hn6uxAA/F4SM1Onxpz92EWv3XF3vCueJMDpKZYXbHYwKFw4l7ZmU= 2021-06-06T05:38:50.850 INFO:teuthology.kill: smithi097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDF7s1uesFb11Ykf8b/KKp//WCihlwMpV3uUXklWgGHMU+FaJ2nvNM6P++AMUe8XsfLGe3N1Am7vcuQQwKJy/0wn255eTcF8eVcZ013Q0XHW/Jt4TIuP9ZQyt2iMZNpYQxPjGsExxxmYUfx1JGTV+sA2uUwVwIXY90WiXdOt44LDArE/Sf8+mohROeQsSIJXnnUf1Q5K/a+cTa19n3dkEbAPv/POrePmRxM9+8jB2sL2CxQE/E6+32eZrGeKA9QMHeWYAjbWgj0pfeRMf3kbn3q+H3M/Zowb/a4P52+C9/Lxz8r5ZtSbWbikY18XrpXqEqTssnqvGgezZn4TJw0PwPC8W0v0e8ss4O7XUyy/RcwipSvTKFl4P5BRDVBdRYWSntp7ZdYvr3qJaUxNC151CFoe5O+Hoa1MHBSoHeiaYJsIfgbCWjj9kbyvU5m/RvftGP1XB1NCVT6E/Mn4GAfCkeqMc5wZb+Fie9UCIHyu9g/r0NYE3ZTOksps+PSXSqjNS0= 2021-06-06T05:38:50.850 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCAOxdJJj0/cTkPTwWwVgIyCCO8GIN+mhZUHP5gSYHH+0nZWJFyD+Ir98C/MYwrBAddbseIgc3tAEPx/BlWWTRTrEKFuBMH+RqeRL29wnVYJvyGAE2JnNcoZxEhAqxwtLK+cFxc6D9n3XC12wVkwSzTPU/HqDR4QZqyKXRK7ul02Kw5TcOWVxXhlhBo9u7SIvPecUEsKNcI2TJsMN1JP1X38FV+DoN1FTi+4R/yYOqCIhz41fd+sdexzzXeRlMvcreCul2+Mi/of2Or8MfmO7Uq9KU8LBkieUa5jgH2k1R3ujThdMpFadVklw57pFsptefvD6tooTutqeMYLHqKlIUtOQpe0wdH+JfU914A/C3hpfZrO9j0NQueTEM0qHtiyvih6H1so8E5e/jaH9IV8RAg189tmfzfJuDFqEtspwwHK4E03hy1DK/eL10ddmWo1ycjDEIAquEl4U+D8TB6+WgyptNmhbMySNRqJlIFLssuG+WG9yw0vEmQtkRkyOAp6c= 2021-06-06T05:38:50.850 INFO:teuthology.kill:2021-06-06 05:38:50,848.848 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-06T05:38:50.851 INFO:teuthology.kill:2021-06-06 05:38:50,851.851 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-06T05:38:50.852 INFO:teuthology.kill:2021-06-06 05:38:50,852.852 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-06T05:38:50.926 INFO:teuthology.kill:2021-06-06 05:38:50,926.926 INFO:teuthology.orchestra.console:Power off smithi153 2021-06-06T05:38:50.941 INFO:teuthology.kill:2021-06-06 05:38:50,941.941 INFO:teuthology.orchestra.console:Power off smithi097 2021-06-06T05:38:50.952 INFO:teuthology.kill:2021-06-06 05:38:50,952.952 INFO:teuthology.orchestra.console:Power off smithi031 2021-06-06T05:38:59.309 INFO:teuthology.kill:2021-06-06 05:38:59,309.309 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-06-06T05:38:59.313 INFO:teuthology.kill:2021-06-06 05:38:59,313.313 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2021-06-06T05:38:59.319 INFO:teuthology.kill:2021-06-06 05:38:59,318.318 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2021-06-06T05:38:59.559 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2021-06-06T05:38:59.560 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2021-06-06T05:38:59.560 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2021-06-06T05:38:59.560 INFO:teuthology.misc:Compressing logs... 2021-06-06T05:38:59.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-06T05:39:59.611 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-06T05:39:59.678 INFO:teuthology.kill:No teuthology processes running 2021-06-06T05:40:00.872 INFO:teuthology.kill:Nuking machines: ['smithi031', 'smithi097', 'smithi153'] 2021-06-06T05:40:01.854 INFO:teuthology.kill:2021-06-06 05:40:01,853.853 INFO:teuthology.nuke:targets: 2021-06-06T05:40:01.854 INFO:teuthology.kill: smithi031.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMUUOdMqODXQ+7gEj5Wn+eMJWOdn1iI4D3X3ZYWS9p/MTw/VaXGlkHvAdBfAeNrYHqkhI1MBN95pVoA7W+fF2SnO/RyeI4HKRVNMrf/qsc8pTpi/crpmRIkfUV7bBxvCisZfMwJbu3KuBsOFhcUonlAXqNB+3Zcm/rXIcaI9Ezzc9+3ObfEDwQVUf75dWEgtw49Jz8+6gvEzEFMz/GyHRVIKxbrjpF5fOG2mF4szQTDWSxh6kmmHqr+K2jGfJJjUpjZVB4Ycwhn9M0EaUKIlc4bK+2UQqhe9uqTmWhiCG48wX0HUq/SBb6ps5wJpuQ02Nd+MjG/KlZJ77MHfCUrD69k8gIjnzj/49RTecmABcEKS6eBX2sJfvXgNFBEuxYKlEPx1t2uKf+4LxCjB8ooZLN6vgjTcdd9Eb5hRvL0PED2xfnQZyGrUgFiUyab4EvkNNkuk4hkG6/HS6hn6uxAA/F4SM1Onxpz92EWv3XF3vCueJMDpKZYXbHYwKFw4l7ZmU= 2021-06-06T05:40:01.855 INFO:teuthology.kill: smithi097.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDF7s1uesFb11Ykf8b/KKp//WCihlwMpV3uUXklWgGHMU+FaJ2nvNM6P++AMUe8XsfLGe3N1Am7vcuQQwKJy/0wn255eTcF8eVcZ013Q0XHW/Jt4TIuP9ZQyt2iMZNpYQxPjGsExxxmYUfx1JGTV+sA2uUwVwIXY90WiXdOt44LDArE/Sf8+mohROeQsSIJXnnUf1Q5K/a+cTa19n3dkEbAPv/POrePmRxM9+8jB2sL2CxQE/E6+32eZrGeKA9QMHeWYAjbWgj0pfeRMf3kbn3q+H3M/Zowb/a4P52+C9/Lxz8r5ZtSbWbikY18XrpXqEqTssnqvGgezZn4TJw0PwPC8W0v0e8ss4O7XUyy/RcwipSvTKFl4P5BRDVBdRYWSntp7ZdYvr3qJaUxNC151CFoe5O+Hoa1MHBSoHeiaYJsIfgbCWjj9kbyvU5m/RvftGP1XB1NCVT6E/Mn4GAfCkeqMc5wZb+Fie9UCIHyu9g/r0NYE3ZTOksps+PSXSqjNS0= 2021-06-06T05:40:01.855 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCAOxdJJj0/cTkPTwWwVgIyCCO8GIN+mhZUHP5gSYHH+0nZWJFyD+Ir98C/MYwrBAddbseIgc3tAEPx/BlWWTRTrEKFuBMH+RqeRL29wnVYJvyGAE2JnNcoZxEhAqxwtLK+cFxc6D9n3XC12wVkwSzTPU/HqDR4QZqyKXRK7ul02Kw5TcOWVxXhlhBo9u7SIvPecUEsKNcI2TJsMN1JP1X38FV+DoN1FTi+4R/yYOqCIhz41fd+sdexzzXeRlMvcreCul2+Mi/of2Or8MfmO7Uq9KU8LBkieUa5jgH2k1R3ujThdMpFadVklw57pFsptefvD6tooTutqeMYLHqKlIUtOQpe0wdH+JfU914A/C3hpfZrO9j0NQueTEM0qHtiyvih6H1so8E5e/jaH9IV8RAg189tmfzfJuDFqEtspwwHK4E03hy1DK/eL10ddmWo1ycjDEIAquEl4U+D8TB6+WgyptNmhbMySNRqJlIFLssuG+WG9yw0vEmQtkRkyOAp6c= 2021-06-06T05:40:01.876 INFO:teuthology.kill:2021-06-06 05:40:01,876.876 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-06T05:40:01.882 INFO:teuthology.kill:2021-06-06 05:40:01,882.882 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-06T05:40:01.887 INFO:teuthology.kill:2021-06-06 05:40:01,887.887 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-06T05:40:01.946 INFO:teuthology.kill:2021-06-06 05:40:01,946.946 INFO:teuthology.orchestra.console:Power off smithi097 2021-06-06T05:40:01.959 INFO:teuthology.kill:2021-06-06 05:40:01,959.959 INFO:teuthology.orchestra.console:Power off smithi031 2021-06-06T05:40:01.968 INFO:teuthology.kill:2021-06-06 05:40:01,968.968 INFO:teuthology.orchestra.console:Power off smithi153 2021-06-06T05:40:02.100 INFO:teuthology.kill:2021-06-06 05:40:02,099.099 INFO:teuthology.orchestra.console:Power off for smithi097 completed 2021-06-06T05:40:02.104 INFO:teuthology.kill:2021-06-06 05:40:02,104.104 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2021-06-06T05:40:02.111 INFO:teuthology.kill:2021-06-06 05:40:02,111.111 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-06-06T05:40:02.242 INFO:teuthology.kill:2021-06-06 05:40:02,241.241 INFO:teuthology.lock.ops:unlocked smithi097.front.sepia.ceph.com 2021-06-06T05:40:02.245 INFO:teuthology.kill:2021-06-06 05:40:02,245.245 INFO:teuthology.lock.ops:unlocked smithi031.front.sepia.ceph.com 2021-06-06T05:40:02.250 INFO:teuthology.kill:2021-06-06 05:40:02,250.250 INFO:teuthology.lock.ops:unlocked smithi153.front.sepia.ceph.com 2021-06-06T05:42:02.516 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-06-06T05:42:02.630 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-06-06T05:42:02.630 INFO:teuthology.nuke:Checking targets against current locks 2021-06-06T05:42:02.697 INFO:teuthology.nuke:Not nuking smithi153.front.sepia.ceph.com because description doesn't match 2021-06-06T05:42:02.697 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 59, in main verbose File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 152, in run_job unlock_targets(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 196, in unlock_targets nuke(fake_ctx, True) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/nuke/__init__.py", line 250, in nuke if ctx.name not in lock['description']: TypeError: argument of type 'NoneType' is not iterable