2021-02-18T01:19:19.866 INFO:root:teuthology version: 1.1.0-62711f33 2021-02-18T01:19:19.979 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi168.front.sepia.ceph.com' reimaging 2021-02-18T01:19:19.980 INFO:teuthology.lock.ops:Updating [ubuntu@smithi168.front.sepia.ceph.com]: reset os type and version on server 2021-02-18T01:19:19.980 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server 2021-02-18T01:19:19.996 INFO:teuthology.lock.ops:Node 'ubuntu@smithi168.front.sepia.ceph.com' reimaging is complete 2021-02-18T01:19:19.997 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi204.front.sepia.ceph.com' reimaging 2021-02-18T01:19:19.997 INFO:teuthology.lock.ops:Updating [ubuntu@smithi204.front.sepia.ceph.com]: reset os type and version on server 2021-02-18T01:19:19.997 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2021-02-18T01:19:20.015 INFO:teuthology.lock.ops:Node 'ubuntu@smithi204.front.sepia.ceph.com' reimaging is complete 2021-02-18T01:19:20.211 INFO:teuthology.provision.fog.smithi168:Scheduling deploy of centos 8.0 2021-02-18T01:19:20.214 INFO:teuthology.provision.fog.smithi204:Scheduling deploy of centos 8.0 2021-02-18T01:19:20.544 INFO:teuthology.orchestra.console:Power off smithi168 2021-02-18T01:19:20.566 INFO:teuthology.orchestra.console:Power off smithi204 2021-02-18T01:19:28.956 INFO:teuthology.orchestra.console:Power off for smithi168 completed 2021-02-18T01:19:28.965 INFO:teuthology.orchestra.console:Power off for smithi204 completed 2021-02-18T01:19:29.060 INFO:teuthology.orchestra.console:Power on smithi168 2021-02-18T01:19:29.072 INFO:teuthology.orchestra.console:Power on smithi204 2021-02-18T01:19:37.446 INFO:teuthology.orchestra.console:Power on for smithi168 completed 2021-02-18T01:19:37.467 INFO:teuthology.orchestra.console:Power on for smithi204 completed 2021-02-18T01:19:37.548 INFO:teuthology.provision.fog.smithi168:Waiting for deploy to finish 2021-02-18T01:19:37.568 INFO:teuthology.provision.fog.smithi204:Waiting for deploy to finish 2021-02-18T01:24:18.270 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-18T01:24:18.609 INFO:teuthology.provision.fog.smithi204:Node is ready 2021-02-18T01:24:18.664 INFO:teuthology.orchestra.run.smithi204.stdout:smithi204.front.sepia.ceph.com 2021-02-18T01:24:18.720 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.15.204 smithi204.front.sepia.ceph.com smithi204 2021-02-18T01:24:19.061 INFO:teuthology.provision.fog.smithi204:Deploy complete! 2021-02-18T01:24:24.385 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-18T01:24:24.739 INFO:teuthology.provision.fog.smithi168:Node is ready 2021-02-18T01:24:24.755 INFO:teuthology.orchestra.run.smithi168.stdout:smithi168.front.sepia.ceph.com 2021-02-18T01:24:24.812 INFO:teuthology.orchestra.run.smithi168.stdout:172.21.15.168 smithi168.front.sepia.ceph.com smithi168 2021-02-18T01:24:25.178 INFO:teuthology.provision.fog.smithi168:Deploy complete! 2021-02-18T01:24:25.339 INFO:teuthology.lock.ops:Checking smithi168.front.sepia.ceph.com 2021-02-18T01:24:25.341 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-18T01:24:25.358 INFO:teuthology.lock.ops:Checking smithi204.front.sepia.ceph.com 2021-02-18T01:24:25.358 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-18T01:24:25.376 INFO:teuthology.lock.ops:Updating [smithi168.front.sepia.ceph.com]: set os type and version on server 2021-02-18T01:24:25.376 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-18T01:24:25.670 INFO:teuthology.orchestra.run.smithi168.stdout:x86_64 2021-02-18T01:24:25.723 INFO:teuthology.orchestra.run.smithi168.stdout:NAME="CentOS Linux" 2021-02-18T01:24:25.724 INFO:teuthology.orchestra.run.smithi168.stdout:VERSION="8 (Core)" 2021-02-18T01:24:25.724 INFO:teuthology.orchestra.run.smithi168.stdout:ID="centos" 2021-02-18T01:24:25.724 INFO:teuthology.orchestra.run.smithi168.stdout:ID_LIKE="rhel fedora" 2021-02-18T01:24:25.724 INFO:teuthology.orchestra.run.smithi168.stdout:VERSION_ID="8" 2021-02-18T01:24:25.725 INFO:teuthology.orchestra.run.smithi168.stdout:PLATFORM_ID="platform:el8" 2021-02-18T01:24:25.725 INFO:teuthology.orchestra.run.smithi168.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-18T01:24:25.725 INFO:teuthology.orchestra.run.smithi168.stdout:ANSI_COLOR="0;31" 2021-02-18T01:24:25.725 INFO:teuthology.orchestra.run.smithi168.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-18T01:24:25.725 INFO:teuthology.orchestra.run.smithi168.stdout:HOME_URL="https://www.centos.org/" 2021-02-18T01:24:25.726 INFO:teuthology.orchestra.run.smithi168.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-18T01:24:25.726 INFO:teuthology.orchestra.run.smithi168.stdout: 2021-02-18T01:24:25.726 INFO:teuthology.orchestra.run.smithi168.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-18T01:24:25.727 INFO:teuthology.orchestra.run.smithi168.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-18T01:24:25.727 INFO:teuthology.orchestra.run.smithi168.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-18T01:24:25.727 INFO:teuthology.orchestra.run.smithi168.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-18T01:24:25.727 INFO:teuthology.orchestra.run.smithi168.stdout: 2021-02-18T01:24:25.729 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server 2021-02-18T01:24:25.747 INFO:teuthology.lock.ops:Updating [smithi204.front.sepia.ceph.com]: set os type and version on server 2021-02-18T01:24:25.747 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-18T01:24:26.046 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2021-02-18T01:24:26.100 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Linux" 2021-02-18T01:24:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8 (Core)" 2021-02-18T01:24:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2021-02-18T01:24:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2021-02-18T01:24:26.101 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8" 2021-02-18T01:24:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8" 2021-02-18T01:24:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-18T01:24:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2021-02-18T01:24:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-18T01:24:26.102 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.centos.org/" 2021-02-18T01:24:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-18T01:24:26.103 INFO:teuthology.orchestra.run.smithi204.stdout: 2021-02-18T01:24:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-18T01:24:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-18T01:24:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-18T01:24:26.104 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-18T01:24:26.104 INFO:teuthology.orchestra.run.smithi204.stdout: 2021-02-18T01:24:26.105 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2021-02-18T01:24:26.173 INFO:teuthology.dispatcher.supervisor:Running job 5891724 2021-02-18T01:24:26.173 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_62711f331901a181e202706309c8fcecdb1e2737/virtualenv/bin/teuthology -v --owner scheduled_mgfritch@teuthology --archive /home/teuthworker/archive/mgfritch-2021-02-18_00:56:34-rados:cephadm-wip-mgfritch-testing-2021-02-17-1029-distro-basic-smithi/5891724 --name mgfritch-2021-02-18_00:56:34-rados:cephadm-wip-mgfritch-testing-2021-02-17-1029-distro-basic-smithi --description rados:cephadm/thrash/{0-distro/centos_8.0 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} -- /home/teuthworker/archive/mgfritch-2021-02-18_00:56:34-rados:cephadm-wip-mgfritch-testing-2021-02-17-1029-distro-basic-smithi/5891724/orig.config.yaml 2021-02-18T01:24:26.176 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mgfritch-2021-02-18_00:56:34-rados:cephadm-wip-mgfritch-testing-2021-02-17-1029-distro-basic-smithi/5891724 2021-02-18T01:24:26.177 INFO:teuthology.dispatcher.supervisor:Job PID: 3129 2021-02-18T01:24:26.177 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-18T13:24:45.741 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-18T13:24:52.479 INFO:teuthology.kill:Killing Pids: {3129} 2021-02-18T13:24:59.151 INFO:teuthology.kill:Nuking machines: ['smithi168', 'smithi204'] 2021-02-18T13:25:04.077 INFO:teuthology.kill:2021-02-18 13:25:04,076.076 INFO:teuthology.nuke:targets: 2021-02-18T13:25:04.084 INFO:teuthology.kill: smithi168.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzLcawtAy3DEFqEFssu5FI3mSC3NHPTMkNit8gW7jpNux64zOKT6dcIzMajmm/n7NLBftm/yDlgnN77mdRKsfuGnncLF5OuIgyr7A6BErg/JwpfdrSVxMWyLfQ1fTix7YI9KIviT9B9VDP/I1XOVZvPy790CVhJyPUzFi0u10vs8cvHVbMrc8uL4scYQeQyd/W4DXo76uEyTWqNEZtcOKPOUDBZPTfvLUgRsvcmcBhWBJsRD/aCVFKVzL0hM10ysVBnImKKkhMFskThnECAjAuEU/EGxSAE5mxMgtrm/x8y3LddfGiuxn4yqpIm7cC4siFbirmDff3DbcoDIiFL0Br+Bkn53GjU51QdiVanZvwYQKZ8ipqUv0bxQ+bKT3ub9/p1r7RfTTeTJoYSHqNN0Q9Oo0yQCJD452J2O06uiqlS/hsefCT7jv+4GneWoL1Hpnu4A+thmCBq93kAQDkqFjMmd2H6B1O0HZWYVvrPwAx4XLFvOWy3cyhjOdtCplqQIU= 2021-02-18T13:25:04.133 INFO:teuthology.kill: smithi204.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCn5BRvtjFkj8qg3v48JDApwXcvgJVIS0eWJDQt+RM1eKNAxAdCKntcuwHGHc0Znfm7kIrE0TJJYpUaUSai77hO+D1AYYN4BqDVjZSukOGRddpPxoi+D+ainITJj6nzTnHNtLuUrehPgTSoQCQdwLmPpcbyLEyEb9FD7W6mc4u8b9ruBpAhIOV15qH5ZlyN+VI22u8/+br/Qn/vA/I4/MWDGCuvKRlKt+BUW+buDQgq4l632fqmJLBKDNq7m2S9Wqog8mRLsz9p4bCKZ1OYi2UtWfwtLdsZv1aBpJ7fX0CB+rgD/jTE+dWV5Ixyaujn3i4nbl4mBm5/GLaQMNIPldLaPO0v4xRnSgZ9wDw2k5PvXpYoJ1KWq9JlfocQ0juFNU9zOs3k3UByRcJ6eg2cbZjDYjOtjnGAielddgT/sz7pq+izCz1zWc3KLMxzDzECT1cibjjMM2HaiaXN07HpSel0m3GVGbSHRI3PnJbqr83hMtZkJodk62sFokWY7r9N+Fk= 2021-02-18T13:25:04.157 INFO:teuthology.kill:2021-02-18 13:25:04,077.077 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-18T13:25:04.160 INFO:teuthology.kill:2021-02-18 13:25:04,160.160 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-18T13:25:04.368 INFO:teuthology.kill:2021-02-18 13:25:04,367.367 INFO:teuthology.orchestra.console:Power off smithi168 2021-02-18T13:25:04.385 INFO:teuthology.kill:2021-02-18 13:25:04,384.384 INFO:teuthology.orchestra.console:Power off smithi204 2021-02-18T13:25:12.755 INFO:teuthology.kill:2021-02-18 13:25:12,754.754 INFO:teuthology.orchestra.console:Power off for smithi168 completed 2021-02-18T13:25:12.858 INFO:teuthology.kill:2021-02-18 13:25:12,858.858 INFO:teuthology.orchestra.console:Power off for smithi204 completed 2021-02-18T13:25:13.590 INFO:teuthology.task.internal:roles: ubuntu@smithi168.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.realm.zone.a', 'node-exporter.a', 'alertmanager.a'] 2021-02-18T13:25:13.631 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-02-18T13:25:13.663 INFO:teuthology.misc:Compressing logs... 2021-02-18T13:25:13.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-18T13:25:48.892 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_62711f331901a181e202706309c8fcecdb1e2737/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_62711f331901a181e202706309c8fcecdb1e2737/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_62711f331901a181e202706309c8fcecdb1e2737/teuthology/misc.py", line 1372, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_62711f331901a181e202706309c8fcecdb1e2737/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_62711f331901a181e202706309c8fcecdb1e2737/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_62711f331901a181e202706309c8fcecdb1e2737/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_62711f331901a181e202706309c8fcecdb1e2737/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-18T13:25:49.159 INFO:teuthology.kill:No teuthology processes running 2021-02-18T13:25:54.850 INFO:teuthology.kill:Nuking machines: ['smithi168', 'smithi204'] 2021-02-18T13:26:00.036 INFO:teuthology.kill:2021-02-18 13:26:00,036.036 INFO:teuthology.nuke:targets: 2021-02-18T13:26:00.041 INFO:teuthology.kill: smithi168.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzLcawtAy3DEFqEFssu5FI3mSC3NHPTMkNit8gW7jpNux64zOKT6dcIzMajmm/n7NLBftm/yDlgnN77mdRKsfuGnncLF5OuIgyr7A6BErg/JwpfdrSVxMWyLfQ1fTix7YI9KIviT9B9VDP/I1XOVZvPy790CVhJyPUzFi0u10vs8cvHVbMrc8uL4scYQeQyd/W4DXo76uEyTWqNEZtcOKPOUDBZPTfvLUgRsvcmcBhWBJsRD/aCVFKVzL0hM10ysVBnImKKkhMFskThnECAjAuEU/EGxSAE5mxMgtrm/x8y3LddfGiuxn4yqpIm7cC4siFbirmDff3DbcoDIiFL0Br+Bkn53GjU51QdiVanZvwYQKZ8ipqUv0bxQ+bKT3ub9/p1r7RfTTeTJoYSHqNN0Q9Oo0yQCJD452J2O06uiqlS/hsefCT7jv+4GneWoL1Hpnu4A+thmCBq93kAQDkqFjMmd2H6B1O0HZWYVvrPwAx4XLFvOWy3cyhjOdtCplqQIU= 2021-02-18T13:26:00.042 INFO:teuthology.kill: smithi204.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCn5BRvtjFkj8qg3v48JDApwXcvgJVIS0eWJDQt+RM1eKNAxAdCKntcuwHGHc0Znfm7kIrE0TJJYpUaUSai77hO+D1AYYN4BqDVjZSukOGRddpPxoi+D+ainITJj6nzTnHNtLuUrehPgTSoQCQdwLmPpcbyLEyEb9FD7W6mc4u8b9ruBpAhIOV15qH5ZlyN+VI22u8/+br/Qn/vA/I4/MWDGCuvKRlKt+BUW+buDQgq4l632fqmJLBKDNq7m2S9Wqog8mRLsz9p4bCKZ1OYi2UtWfwtLdsZv1aBpJ7fX0CB+rgD/jTE+dWV5Ixyaujn3i4nbl4mBm5/GLaQMNIPldLaPO0v4xRnSgZ9wDw2k5PvXpYoJ1KWq9JlfocQ0juFNU9zOs3k3UByRcJ6eg2cbZjDYjOtjnGAielddgT/sz7pq+izCz1zWc3KLMxzDzECT1cibjjMM2HaiaXN07HpSel0m3GVGbSHRI3PnJbqr83hMtZkJodk62sFokWY7r9N+Fk= 2021-02-18T13:26:00.070 INFO:teuthology.kill:2021-02-18 13:26:00,069.069 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-18T13:26:00.075 INFO:teuthology.kill:2021-02-18 13:26:00,074.074 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-18T13:26:00.155 INFO:teuthology.kill:2021-02-18 13:26:00,155.155 INFO:teuthology.orchestra.console:Power off smithi168 2021-02-18T13:26:00.177 INFO:teuthology.kill:2021-02-18 13:26:00,176.176 INFO:teuthology.orchestra.console:Power off smithi204 2021-02-18T13:26:00.361 INFO:teuthology.kill:2021-02-18 13:26:00,360.360 INFO:teuthology.orchestra.console:Power off for smithi168 completed 2021-02-18T13:26:00.365 INFO:teuthology.kill:2021-02-18 13:26:00,365.365 INFO:teuthology.orchestra.console:Power off for smithi204 completed 2021-02-18T13:26:00.517 INFO:teuthology.kill:2021-02-18 13:26:00,516.516 INFO:teuthology.lock.ops:unlocked smithi168.front.sepia.ceph.com 2021-02-18T13:26:00.520 INFO:teuthology.kill:2021-02-18 13:26:00,520.520 INFO:teuthology.lock.ops:unlocked smithi204.front.sepia.ceph.com 2021-02-18T13:28:00.753 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-18T13:28:02.053 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-18T13:28:02.056 INFO:teuthology.nuke:Checking targets against current locks 2021-02-18T13:28:02.376 INFO:teuthology.nuke:Not nuking smithi168.front.sepia.ceph.com because description doesn't match 2021-02-18T13:28:02.377 INFO:teuthology.nuke:Not nuking smithi204.front.sepia.ceph.com because description doesn't match