2021-03-13T08:37:52.632 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-13T08:37:52.732 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging 2021-03-13T08:37:52.733 INFO:teuthology.lock.ops:Updating [ubuntu@smithi153.front.sepia.ceph.com]: reset os type and version on server 2021-03-13T08:37:52.733 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2021-03-13T08:37:52.748 INFO:teuthology.lock.ops:Node 'ubuntu@smithi153.front.sepia.ceph.com' reimaging is complete 2021-03-13T08:37:52.749 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi199.front.sepia.ceph.com' reimaging 2021-03-13T08:37:52.749 INFO:teuthology.lock.ops:Updating [ubuntu@smithi199.front.sepia.ceph.com]: reset os type and version on server 2021-03-13T08:37:52.749 INFO:teuthology.lock.ops:Updating smithi199.front.sepia.ceph.com on lock server 2021-03-13T08:37:52.765 INFO:teuthology.lock.ops:Node 'ubuntu@smithi199.front.sepia.ceph.com' reimaging is complete 2021-03-13T08:37:52.961 INFO:teuthology.provision.fog.smithi199:Scheduling deploy of rhel 8.3 2021-03-13T08:37:52.965 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of rhel 8.3 2021-03-13T08:37:53.293 INFO:teuthology.orchestra.console:Power off smithi199 2021-03-13T08:37:53.307 INFO:teuthology.orchestra.console:Power off smithi153 2021-03-13T08:38:01.664 INFO:teuthology.orchestra.console:Power off for smithi199 completed 2021-03-13T08:38:01.669 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-03-13T08:38:01.766 INFO:teuthology.orchestra.console:Power on smithi199 2021-03-13T08:38:01.773 INFO:teuthology.orchestra.console:Power on smithi153 2021-03-13T08:38:10.129 INFO:teuthology.orchestra.console:Power on for smithi199 completed 2021-03-13T08:38:10.134 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2021-03-13T08:38:10.231 INFO:teuthology.provision.fog.smithi199:Waiting for deploy to finish 2021-03-13T08:38:10.235 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2021-03-13T08:42:22.495 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-13T08:42:37.997 INFO:teuthology.provision.fog.smithi199:Node is ready 2021-03-13T08:42:38.016 INFO:teuthology.orchestra.run.smithi199.stdout:smithi087 2021-03-13T08:42:38.072 INFO:teuthology.orchestra.run.smithi199.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2021-03-13T08:42:38.439 INFO:teuthology.provision.fog.smithi199:Deploy complete! 2021-03-13T08:42:57.103 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-13T08:42:57.398 INFO:teuthology.provision.fog.smithi153:Node is ready 2021-03-13T08:42:57.412 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2021-03-13T08:42:57.466 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2021-03-13T08:42:57.783 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2021-03-13T08:42:57.933 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2021-03-13T08:42:57.934 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-13T08:42:57.950 INFO:teuthology.lock.ops:Checking smithi199.front.sepia.ceph.com 2021-03-13T08:42:57.950 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-13T08:42:57.964 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2021-03-13T08:42:57.964 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T08:42:58.258 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2021-03-13T08:42:58.330 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Red Hat Enterprise Linux" 2021-03-13T08:42:58.330 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8.3 (Ootpa)" 2021-03-13T08:42:58.331 INFO:teuthology.orchestra.run.smithi153.stdout:ID="rhel" 2021-03-13T08:42:58.331 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="fedora" 2021-03-13T08:42:58.331 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8.3" 2021-03-13T08:42:58.331 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2021-03-13T08:42:58.332 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-03-13T08:42:58.332 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2021-03-13T08:42:58.332 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-03-13T08:42:58.332 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.redhat.com/" 2021-03-13T08:42:58.332 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-03-13T08:42:58.333 INFO:teuthology.orchestra.run.smithi153.stdout: 2021-03-13T08:42:58.333 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-03-13T08:42:58.333 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-03-13T08:42:58.333 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-03-13T08:42:58.333 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-03-13T08:42:58.335 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2021-03-13T08:42:58.356 INFO:teuthology.lock.ops:Updating [smithi199.front.sepia.ceph.com]: set os type and version on server 2021-03-13T08:42:58.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T08:42:58.733 INFO:teuthology.orchestra.run.smithi199.stdout:x86_64 2021-03-13T08:42:58.805 INFO:teuthology.orchestra.run.smithi199.stdout:NAME="Red Hat Enterprise Linux" 2021-03-13T08:42:58.806 INFO:teuthology.orchestra.run.smithi199.stdout:VERSION="8.3 (Ootpa)" 2021-03-13T08:42:58.806 INFO:teuthology.orchestra.run.smithi199.stdout:ID="rhel" 2021-03-13T08:42:58.806 INFO:teuthology.orchestra.run.smithi199.stdout:ID_LIKE="fedora" 2021-03-13T08:42:58.806 INFO:teuthology.orchestra.run.smithi199.stdout:VERSION_ID="8.3" 2021-03-13T08:42:58.807 INFO:teuthology.orchestra.run.smithi199.stdout:PLATFORM_ID="platform:el8" 2021-03-13T08:42:58.807 INFO:teuthology.orchestra.run.smithi199.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-03-13T08:42:58.807 INFO:teuthology.orchestra.run.smithi199.stdout:ANSI_COLOR="0;31" 2021-03-13T08:42:58.807 INFO:teuthology.orchestra.run.smithi199.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-03-13T08:42:58.807 INFO:teuthology.orchestra.run.smithi199.stdout:HOME_URL="https://www.redhat.com/" 2021-03-13T08:42:58.808 INFO:teuthology.orchestra.run.smithi199.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-03-13T08:42:58.808 INFO:teuthology.orchestra.run.smithi199.stdout: 2021-03-13T08:42:58.808 INFO:teuthology.orchestra.run.smithi199.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-03-13T08:42:58.808 INFO:teuthology.orchestra.run.smithi199.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-03-13T08:42:58.808 INFO:teuthology.orchestra.run.smithi199.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-03-13T08:42:58.809 INFO:teuthology.orchestra.run.smithi199.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-03-13T08:42:58.810 INFO:teuthology.lock.ops:Updating smithi199.front.sepia.ceph.com on lock server 2021-03-13T08:42:58.901 INFO:teuthology.dispatcher.supervisor:Running job 5961031 2021-03-13T08:42:58.902 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/bin/teuthology -v --owner scheduled_kchai@teuthology --archive /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961031 --name kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi --description rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/repair_test} -- /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961031/orig.config.yaml 2021-03-13T08:42:58.905 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/kchai-2021-03-13_08:16:32-rados-wip-kefu-testing-2021-03-13-1449-distro-basic-smithi/5961031 2021-03-13T08:42:58.907 INFO:teuthology.dispatcher.supervisor:Job PID: 1018 2021-03-13T08:42:58.907 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-13T20:43:50.711 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-13T20:43:51.002 INFO:teuthology.kill:Killing Pids: {1018} 2021-03-13T20:43:52.396 INFO:teuthology.kill:Nuking machines: ['smithi153', 'smithi199'] 2021-03-13T20:43:54.063 INFO:teuthology.kill:2021-03-13 20:43:54,063.063 INFO:teuthology.nuke:targets: 2021-03-13T20:43:54.064 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2Hes6BLuDo8x7rTZ5qumGntQ0V3d3Fn/2NP2aMGHyTWBWhRMwrolqBagBHq5qz/HweYlozxnRV0baT/QWVB08M0o67ssU6yF1j+v150woZGKWN9F+l9Apj5Xh3RUmkX/YWfUMG+oZWnkvEdwpb6Jm1ZIZ2gyWCXobpYRjZWLDKHibHsYtb5zzKrgjwueT6eOLzCaEncbJ2lBf3QZmZAoiq4vO3JB4Y+u6Iz5a42kzzMoAqaxkpt9TOHpw5xNeskbnH7p+PeQyMShB64T7VTw+UijsKImeNvkgrjjozQd8Iu1ZD3wLmB4kl+T8CBdxUwv54J/Hgqm6qVasKrmhYdUvHK0Y36Cvik7NWtbcc01TUnEWOGCR8Se8kllg2vlEmAQtvU3yA4604nOjMNZiIn1GInCK1BTz4IQadZat/c1WZyiK5eLK3e40AhCOomvUw00uUL7cgnmPWC5m5lckxkD4zW8fkuOLpSLo5xotQ0zDjiqPMGAL5QEWAQk9TZwl3K8= 2021-03-13T20:43:54.064 INFO:teuthology.kill: smithi199.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwE536VcdngqcvPtp/H+8We+hV8GB3t41v8Hx7se7+iYkki6PeKfd5WuVARFX6cPcY4vdeHB/5lkZr6CJCFfOAmjT32Xcu5KaEeLA3IkW2CczzxI7m67tCgb1gOVgnC7FO946K4Ppdc9WG0CLAWEKQOui+8b4rd7r4Vm9LMmFxGr+ZlHGi/xPlscZb9YGtm/G8U65/sB5biawF2uFJb4+B0zktzuS6hwOKFvjGmQTrd/ctDkicRT2YjteW6mbbkGmQNZr3yQfhhdHDkrFNJDFqQoajY0uhbV4RTRjhzV+dZqy3aS3qMNR4IuXVcStFmyVSuTZ800+eIAH1kfFK+mfwY2z5BnN4SJIQCJo/3zcdQGkIUK7X+3ICaPmHiZyGvcqsIEbk94ID/lMVE65oaSRknc/iqL+CKOd00QJ2EEzEbo4KE2Vq5LI1+hfnWwbslOEvEfw238tJHMbuoNN9kkEBRZEX23fsI5sTFGYQyLpP3FRunF4h1QLNA+h9fsRwVbs= 2021-03-13T20:43:54.065 INFO:teuthology.kill:2021-03-13 20:43:54,063.063 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T20:43:54.066 INFO:teuthology.kill:2021-03-13 20:43:54,066.066 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T20:43:54.176 INFO:teuthology.kill:2021-03-13 20:43:54,175.175 INFO:teuthology.orchestra.console:Power off smithi153 2021-03-13T20:43:54.244 INFO:teuthology.kill:2021-03-13 20:43:54,243.243 INFO:teuthology.orchestra.console:Power off smithi199 2021-03-13T20:44:02.674 INFO:teuthology.kill:2021-03-13 20:44:02,674.674 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-03-13T20:44:02.697 INFO:teuthology.kill:2021-03-13 20:44:02,697.697 INFO:teuthology.orchestra.console:Power off for smithi199 completed 2021-03-13T20:44:02.922 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2021-03-13T20:44:02.923 INFO:teuthology.task.internal:roles: ubuntu@smithi199.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-03-13T20:44:02.923 INFO:teuthology.misc:Compressing logs... 2021-03-13T20:44:02.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-13T20:44:21.319 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-13T20:44:21.491 INFO:teuthology.kill:No teuthology processes running 2021-03-13T20:44:23.101 INFO:teuthology.kill:Nuking machines: ['smithi153', 'smithi199'] 2021-03-13T20:44:24.543 INFO:teuthology.kill:2021-03-13 20:44:24,543.543 INFO:teuthology.nuke:targets: 2021-03-13T20:44:24.544 INFO:teuthology.kill: smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2Hes6BLuDo8x7rTZ5qumGntQ0V3d3Fn/2NP2aMGHyTWBWhRMwrolqBagBHq5qz/HweYlozxnRV0baT/QWVB08M0o67ssU6yF1j+v150woZGKWN9F+l9Apj5Xh3RUmkX/YWfUMG+oZWnkvEdwpb6Jm1ZIZ2gyWCXobpYRjZWLDKHibHsYtb5zzKrgjwueT6eOLzCaEncbJ2lBf3QZmZAoiq4vO3JB4Y+u6Iz5a42kzzMoAqaxkpt9TOHpw5xNeskbnH7p+PeQyMShB64T7VTw+UijsKImeNvkgrjjozQd8Iu1ZD3wLmB4kl+T8CBdxUwv54J/Hgqm6qVasKrmhYdUvHK0Y36Cvik7NWtbcc01TUnEWOGCR8Se8kllg2vlEmAQtvU3yA4604nOjMNZiIn1GInCK1BTz4IQadZat/c1WZyiK5eLK3e40AhCOomvUw00uUL7cgnmPWC5m5lckxkD4zW8fkuOLpSLo5xotQ0zDjiqPMGAL5QEWAQk9TZwl3K8= 2021-03-13T20:44:24.544 INFO:teuthology.kill: smithi199.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwE536VcdngqcvPtp/H+8We+hV8GB3t41v8Hx7se7+iYkki6PeKfd5WuVARFX6cPcY4vdeHB/5lkZr6CJCFfOAmjT32Xcu5KaEeLA3IkW2CczzxI7m67tCgb1gOVgnC7FO946K4Ppdc9WG0CLAWEKQOui+8b4rd7r4Vm9LMmFxGr+ZlHGi/xPlscZb9YGtm/G8U65/sB5biawF2uFJb4+B0zktzuS6hwOKFvjGmQTrd/ctDkicRT2YjteW6mbbkGmQNZr3yQfhhdHDkrFNJDFqQoajY0uhbV4RTRjhzV+dZqy3aS3qMNR4IuXVcStFmyVSuTZ800+eIAH1kfFK+mfwY2z5BnN4SJIQCJo/3zcdQGkIUK7X+3ICaPmHiZyGvcqsIEbk94ID/lMVE65oaSRknc/iqL+CKOd00QJ2EEzEbo4KE2Vq5LI1+hfnWwbslOEvEfw238tJHMbuoNN9kkEBRZEX23fsI5sTFGYQyLpP3FRunF4h1QLNA+h9fsRwVbs= 2021-03-13T20:44:24.570 INFO:teuthology.kill:2021-03-13 20:44:24,570.570 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T20:44:24.573 INFO:teuthology.kill:2021-03-13 20:44:24,572.572 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-13T20:44:24.661 INFO:teuthology.kill:2021-03-13 20:44:24,661.661 INFO:teuthology.orchestra.console:Power off smithi199 2021-03-13T20:44:24.728 INFO:teuthology.kill:2021-03-13 20:44:24,727.727 INFO:teuthology.orchestra.console:Power off smithi153 2021-03-13T20:44:24.926 INFO:teuthology.kill:2021-03-13 20:44:24,926.926 INFO:teuthology.orchestra.console:Power off for smithi199 completed 2021-03-13T20:44:24.951 INFO:teuthology.kill:2021-03-13 20:44:24,951.951 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2021-03-13T20:44:25.066 INFO:teuthology.kill:2021-03-13 20:44:25,065.065 INFO:teuthology.lock.ops:unlocked smithi199.front.sepia.ceph.com 2021-03-13T20:44:25.100 INFO:teuthology.kill:2021-03-13 20:44:25,094.094 INFO:teuthology.lock.ops:unlocked smithi153.front.sepia.ceph.com 2021-03-13T20:46:25.459 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-03-13T20:46:25.622 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-03-13T20:46:25.623 INFO:teuthology.nuke:Checking targets against current locks 2021-03-13T20:46:25.720 INFO:teuthology.nuke:Not nuking smithi153.front.sepia.ceph.com because description doesn't match 2021-03-13T20:46:25.721 INFO:teuthology.nuke:Not nuking smithi199.front.sepia.ceph.com because description doesn't match