2021-05-15T10:26:03.170 INFO:root:teuthology version: 1.1.0-19220a3b 2021-05-15T10:26:03.279 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging 2021-05-15T10:26:03.280 INFO:teuthology.lock.ops:Updating [ubuntu@smithi057.front.sepia.ceph.com]: reset os type and version on server 2021-05-15T10:26:03.280 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2021-05-15T10:26:03.294 INFO:teuthology.lock.ops:Node 'ubuntu@smithi057.front.sepia.ceph.com' reimaging is complete 2021-05-15T10:26:03.294 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi105.front.sepia.ceph.com' reimaging 2021-05-15T10:26:03.294 INFO:teuthology.lock.ops:Updating [ubuntu@smithi105.front.sepia.ceph.com]: reset os type and version on server 2021-05-15T10:26:03.294 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2021-05-15T10:26:03.308 INFO:teuthology.lock.ops:Node 'ubuntu@smithi105.front.sepia.ceph.com' reimaging is complete 2021-05-15T10:26:03.549 INFO:teuthology.provision.fog.smithi057:Scheduling deploy of centos 8.2 2021-05-15T10:26:03.585 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of centos 8.2 2021-05-15T10:26:04.000 INFO:teuthology.orchestra.console:Power off smithi057 2021-05-15T10:26:04.055 INFO:teuthology.orchestra.console:Power off smithi105 2021-05-15T10:26:12.372 INFO:teuthology.orchestra.console:Power off for smithi057 completed 2021-05-15T10:26:12.473 INFO:teuthology.orchestra.console:Power on smithi057 2021-05-15T10:26:16.528 INFO:teuthology.orchestra.console:Power off for smithi105 completed 2021-05-15T10:26:16.630 INFO:teuthology.orchestra.console:Power on smithi105 2021-05-15T10:26:20.930 INFO:teuthology.orchestra.console:Power on for smithi057 completed 2021-05-15T10:26:21.031 INFO:teuthology.provision.fog.smithi057:Waiting for deploy to finish 2021-05-15T10:26:25.061 INFO:teuthology.orchestra.console:Power on for smithi105 completed 2021-05-15T10:26:25.162 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish 2021-05-15T10:30:53.170 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-15T10:30:53.489 INFO:teuthology.provision.fog.smithi057:Node is ready 2021-05-15T10:30:53.505 INFO:teuthology.orchestra.run.smithi057.stdout:smithi057.front.sepia.ceph.com 2021-05-15T10:30:53.560 INFO:teuthology.orchestra.run.smithi057.stdout:172.21.15.57 smithi057.front.sepia.ceph.com smithi057 2021-05-15T10:30:53.894 INFO:teuthology.provision.fog.smithi057:Deploy complete! 2021-05-15T10:31:06.078 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-15T10:31:06.412 INFO:teuthology.provision.fog.smithi105:Node is ready 2021-05-15T10:31:06.427 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105.front.sepia.ceph.com 2021-05-15T10:31:06.484 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105 smithi105.front.sepia.ceph.com smithi105 2021-05-15T10:31:06.802 INFO:teuthology.provision.fog.smithi105:Deploy complete! 2021-05-15T10:31:06.942 INFO:teuthology.lock.ops:Checking smithi057.front.sepia.ceph.com 2021-05-15T10:31:06.943 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-15T10:31:06.967 INFO:teuthology.lock.ops:Checking smithi105.front.sepia.ceph.com 2021-05-15T10:31:06.967 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-15T10:31:06.981 INFO:teuthology.lock.ops:Updating [smithi057.front.sepia.ceph.com]: set os type and version on server 2021-05-15T10:31:06.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-15T10:31:07.268 INFO:teuthology.orchestra.run.smithi057.stdout:x86_64 2021-05-15T10:31:07.323 INFO:teuthology.orchestra.run.smithi057.stdout:NAME="CentOS Linux" 2021-05-15T10:31:07.324 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION="8 (Core)" 2021-05-15T10:31:07.324 INFO:teuthology.orchestra.run.smithi057.stdout:ID="centos" 2021-05-15T10:31:07.325 INFO:teuthology.orchestra.run.smithi057.stdout:ID_LIKE="rhel fedora" 2021-05-15T10:31:07.325 INFO:teuthology.orchestra.run.smithi057.stdout:VERSION_ID="8" 2021-05-15T10:31:07.325 INFO:teuthology.orchestra.run.smithi057.stdout:PLATFORM_ID="platform:el8" 2021-05-15T10:31:07.326 INFO:teuthology.orchestra.run.smithi057.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-05-15T10:31:07.326 INFO:teuthology.orchestra.run.smithi057.stdout:ANSI_COLOR="0;31" 2021-05-15T10:31:07.326 INFO:teuthology.orchestra.run.smithi057.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-05-15T10:31:07.326 INFO:teuthology.orchestra.run.smithi057.stdout:HOME_URL="https://www.centos.org/" 2021-05-15T10:31:07.327 INFO:teuthology.orchestra.run.smithi057.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-05-15T10:31:07.327 INFO:teuthology.orchestra.run.smithi057.stdout: 2021-05-15T10:31:07.327 INFO:teuthology.orchestra.run.smithi057.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-05-15T10:31:07.327 INFO:teuthology.orchestra.run.smithi057.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-05-15T10:31:07.327 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-05-15T10:31:07.328 INFO:teuthology.orchestra.run.smithi057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-05-15T10:31:07.328 INFO:teuthology.orchestra.run.smithi057.stdout: 2021-05-15T10:31:07.329 INFO:teuthology.lock.ops:Updating smithi057.front.sepia.ceph.com on lock server 2021-05-15T10:31:07.345 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: set os type and version on server 2021-05-15T10:31:07.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-15T10:31:07.606 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2021-05-15T10:31:07.660 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Linux" 2021-05-15T10:31:07.661 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8 (Core)" 2021-05-15T10:31:07.661 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2021-05-15T10:31:07.661 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2021-05-15T10:31:07.661 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8" 2021-05-15T10:31:07.661 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2021-05-15T10:31:07.662 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-05-15T10:31:07.662 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2021-05-15T10:31:07.662 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-05-15T10:31:07.662 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.centos.org/" 2021-05-15T10:31:07.663 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-05-15T10:31:07.663 INFO:teuthology.orchestra.run.smithi105.stdout: 2021-05-15T10:31:07.663 INFO:teuthology.orchestra.run.smithi105.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-05-15T10:31:07.663 INFO:teuthology.orchestra.run.smithi105.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-05-15T10:31:07.663 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-05-15T10:31:07.664 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-05-15T10:31:07.664 INFO:teuthology.orchestra.run.smithi105.stdout: 2021-05-15T10:31:07.664 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2021-05-15T10:31:07.766 INFO:teuthology.dispatcher.supervisor:Running job 6115982 2021-05-15T10:31:07.767 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/virtualenv/bin/teuthology -v --owner scheduled_kchai@teuthology --archive /home/teuthworker/archive/kchai-2021-05-15_10:00:36-rados-wip-kefu-testing-2021-05-15-1545-distro-basic-smithi/6115982 --name kchai-2021-05-15_10:00:36-rados-wip-kefu-testing-2021-05-15-1545-distro-basic-smithi --description rados/cephadm/thrash/{0-distro/centos_8.2_kubic_stable 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} -- /home/teuthworker/archive/kchai-2021-05-15_10:00:36-rados-wip-kefu-testing-2021-05-15-1545-distro-basic-smithi/6115982/orig.config.yaml 2021-05-15T10:31:07.770 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/kchai-2021-05-15_10:00:36-rados-wip-kefu-testing-2021-05-15-1545-distro-basic-smithi/6115982 2021-05-15T10:31:07.771 INFO:teuthology.dispatcher.supervisor:Job PID: 26275 2021-05-15T10:31:07.771 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-05-15T22:31:55.141 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-05-15T22:31:55.302 INFO:teuthology.kill:Killing Pids: {26275} 2021-05-15T22:31:56.292 INFO:teuthology.kill:Nuking machines: ['smithi057', 'smithi105'] 2021-05-15T22:31:57.154 INFO:teuthology.kill:2021-05-15 22:31:57,154.154 INFO:teuthology.nuke:targets: 2021-05-15T22:31:57.155 INFO:teuthology.kill: smithi057.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0JDM2QBHT2RzS2yXGgtfBXwGkKIaS2Be/qSu2KFgpb253/zW/Lz1aanYnVL3nflH8muB1y1ex+4rliG8tNAE2KzlJSVo5dXfggMRGsiwfyrjlOrhS+WwKdk0mJjnheR+RrPbXOlr0MHwva4Qe1eDzko+TxH2qyjxO3+1dn+Mi2A6TzXaXq8dQx4Ee8SgoaU52Dgj7dDmE3nmOLyccdQIG5445U5bu3sV/7G6IZkxZv3DfHlZCdqItpttvM4Uo5TVAKaRfuba+Nxwp/PWBulJHwD6RnKIjljC+aFX3++Yqb1d45JWHUxL85PZXoRaz0fRt7ZZ0vS1wJZ4PiJoa8u1LwXdtSf8w7qgeM0rfSa5e/yxQRlgsBxb3eupsKuodBQhZF8xg9/R8yIAMkecmC/94n/0zqNJxJMBMHUthduV5a0Dt4WglY7ImDXiVSHXV2sRKoP8jtCfg51GjIHwyskYLOF5o+pXePv8tkobjgiag77zINod1SfhB/cL41pBrEv8= 2021-05-15T22:31:57.156 INFO:teuthology.kill: smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9oC3iBRebtYbXIg/q3wSrRzWW78bxKz+oIwYSJZ/KT2LBfiSNyxdfIWzgiEXrPf1cRmgBRmw5OiZv8DLx92WrOt6JyjZK1B4sE6TlEx8qzs1VCoPFYQ0G4oI7UtBMBVfhLxWMTrrz8mxfor9BfiBoXY4kuLzWUgeJ9mvNboZGcSN79BTM6NO52JjzC17/Xse+5lGUAq71oTekRWMLkuRnUznAi3GUHnyS7crxFaKboXiOxzhFSKqVUiGW3GOm8e06SrN84QNPpFvoadA1aEG95wWS3maSyjAjwLCH4vuYGxa+Pvma6xe4W14ewPFM3NXir1dpoPV5WIwULmYsEjC2sw0cSgxLQCsKSi+ZXrZ4xzx23OveI88WE6s36Ya2jDSAkHM8moBdR3X/X3mCuLlg0GMaZh/MkzCvRAvhqcATNmotI6pplNyIw4dDRnHNgJpJTF01fzgymI+grqjIFje49tDFLoDFeI+CsDvqJLixS8oMbT7BOEmorQagWVn2t5k= 2021-05-15T22:31:57.156 INFO:teuthology.kill:2021-05-15 22:31:57,154.154 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-15T22:31:57.157 INFO:teuthology.kill:2021-05-15 22:31:57,157.157 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-15T22:31:57.212 INFO:teuthology.kill:2021-05-15 22:31:57,212.212 INFO:teuthology.orchestra.console:Power off smithi057 2021-05-15T22:31:57.223 INFO:teuthology.kill:2021-05-15 22:31:57,223.223 INFO:teuthology.orchestra.console:Power off smithi105 2021-05-15T22:32:05.594 INFO:teuthology.kill:2021-05-15 22:32:05,594.594 INFO:teuthology.orchestra.console:Power off for smithi105 completed 2021-05-15T22:32:09.702 INFO:teuthology.kill:2021-05-15 22:32:09,702.702 INFO:teuthology.orchestra.console:Power off for smithi057 completed 2021-05-15T22:32:09.907 INFO:teuthology.task.internal:roles: ubuntu@smithi057.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2021-05-15T22:32:09.908 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-05-15T22:32:09.908 INFO:teuthology.misc:Compressing logs... 2021-05-15T22:32:09.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-15T22:32:44.491 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-05-15T22:32:44.541 INFO:teuthology.kill:No teuthology processes running 2021-05-15T22:32:45.509 INFO:teuthology.kill:Nuking machines: ['smithi057', 'smithi105'] 2021-05-15T22:32:46.355 INFO:teuthology.kill:2021-05-15 22:32:46,354.354 INFO:teuthology.nuke:targets: 2021-05-15T22:32:46.355 INFO:teuthology.kill: smithi057.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0JDM2QBHT2RzS2yXGgtfBXwGkKIaS2Be/qSu2KFgpb253/zW/Lz1aanYnVL3nflH8muB1y1ex+4rliG8tNAE2KzlJSVo5dXfggMRGsiwfyrjlOrhS+WwKdk0mJjnheR+RrPbXOlr0MHwva4Qe1eDzko+TxH2qyjxO3+1dn+Mi2A6TzXaXq8dQx4Ee8SgoaU52Dgj7dDmE3nmOLyccdQIG5445U5bu3sV/7G6IZkxZv3DfHlZCdqItpttvM4Uo5TVAKaRfuba+Nxwp/PWBulJHwD6RnKIjljC+aFX3++Yqb1d45JWHUxL85PZXoRaz0fRt7ZZ0vS1wJZ4PiJoa8u1LwXdtSf8w7qgeM0rfSa5e/yxQRlgsBxb3eupsKuodBQhZF8xg9/R8yIAMkecmC/94n/0zqNJxJMBMHUthduV5a0Dt4WglY7ImDXiVSHXV2sRKoP8jtCfg51GjIHwyskYLOF5o+pXePv8tkobjgiag77zINod1SfhB/cL41pBrEv8= 2021-05-15T22:32:46.356 INFO:teuthology.kill: smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC9oC3iBRebtYbXIg/q3wSrRzWW78bxKz+oIwYSJZ/KT2LBfiSNyxdfIWzgiEXrPf1cRmgBRmw5OiZv8DLx92WrOt6JyjZK1B4sE6TlEx8qzs1VCoPFYQ0G4oI7UtBMBVfhLxWMTrrz8mxfor9BfiBoXY4kuLzWUgeJ9mvNboZGcSN79BTM6NO52JjzC17/Xse+5lGUAq71oTekRWMLkuRnUznAi3GUHnyS7crxFaKboXiOxzhFSKqVUiGW3GOm8e06SrN84QNPpFvoadA1aEG95wWS3maSyjAjwLCH4vuYGxa+Pvma6xe4W14ewPFM3NXir1dpoPV5WIwULmYsEjC2sw0cSgxLQCsKSi+ZXrZ4xzx23OveI88WE6s36Ya2jDSAkHM8moBdR3X/X3mCuLlg0GMaZh/MkzCvRAvhqcATNmotI6pplNyIw4dDRnHNgJpJTF01fzgymI+grqjIFje49tDFLoDFeI+CsDvqJLixS8oMbT7BOEmorQagWVn2t5k= 2021-05-15T22:32:46.374 INFO:teuthology.kill:2021-05-15 22:32:46,374.374 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-15T22:32:46.377 INFO:teuthology.kill:2021-05-15 22:32:46,377.377 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-15T22:32:46.428 INFO:teuthology.kill:2021-05-15 22:32:46,428.428 INFO:teuthology.orchestra.console:Power off smithi105 2021-05-15T22:32:46.440 INFO:teuthology.kill:2021-05-15 22:32:46,440.440 INFO:teuthology.orchestra.console:Power off smithi057 2021-05-15T22:32:46.566 INFO:teuthology.kill:2021-05-15 22:32:46,566.566 INFO:teuthology.orchestra.console:Power off for smithi105 completed 2021-05-15T22:32:46.577 INFO:teuthology.kill:2021-05-15 22:32:46,577.577 INFO:teuthology.orchestra.console:Power off for smithi057 completed 2021-05-15T22:32:46.707 INFO:teuthology.kill:2021-05-15 22:32:46,707.707 INFO:teuthology.lock.ops:unlocked smithi105.front.sepia.ceph.com 2021-05-15T22:32:46.714 INFO:teuthology.kill:2021-05-15 22:32:46,714.714 INFO:teuthology.lock.ops:unlocked smithi057.front.sepia.ceph.com 2021-05-15T22:34:46.982 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15