2021-02-15T21:43:23.875 INFO:root:teuthology version: 1.1.0-64d5f180
2021-02-15T21:43:23.997 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi115.front.sepia.ceph.com' reimaging
2021-02-15T21:43:23.997 INFO:teuthology.lock.ops:Updating [ubuntu@smithi115.front.sepia.ceph.com]: reset os type and version on server
2021-02-15T21:43:23.997 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server
2021-02-15T21:43:24.014 INFO:teuthology.lock.ops:Node 'ubuntu@smithi115.front.sepia.ceph.com' reimaging is complete
2021-02-15T21:43:24.014 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging
2021-02-15T21:43:24.014 INFO:teuthology.lock.ops:Updating [ubuntu@smithi150.front.sepia.ceph.com]: reset os type and version on server
2021-02-15T21:43:24.015 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server
2021-02-15T21:43:24.038 INFO:teuthology.lock.ops:Node 'ubuntu@smithi150.front.sepia.ceph.com' reimaging is complete
2021-02-15T21:43:24.232 INFO:teuthology.provision.fog.smithi115:Scheduling deploy of centos 8.2
2021-02-15T21:43:24.235 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of centos 8.2
2021-02-15T21:43:24.595 INFO:teuthology.orchestra.console:Power off smithi150
2021-02-15T21:43:24.624 INFO:teuthology.orchestra.console:Power off smithi115
2021-02-15T21:43:33.004 INFO:teuthology.orchestra.console:Power off for smithi150 completed
2021-02-15T21:43:33.105 INFO:teuthology.orchestra.console:Power on smithi150
2021-02-15T21:43:37.136 INFO:teuthology.orchestra.console:Power off for smithi115 completed
2021-02-15T21:43:37.238 INFO:teuthology.orchestra.console:Power on smithi115
2021-02-15T21:43:41.549 INFO:teuthology.orchestra.console:Power on for smithi115 completed
2021-02-15T21:43:41.650 INFO:teuthology.provision.fog.smithi115:Waiting for deploy to finish
2021-02-15T21:43:41.711 INFO:teuthology.orchestra.console:Power on for smithi150 completed
2021-02-15T21:43:41.812 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish
2021-02-15T21:46:12.962 ERROR:teuthology.orchestra.connection:Error authenticating with smithi150.front.sepia.ceph.com: Authentication failed.
2021-02-15T21:48:03.144 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-02-15T21:48:23.487 INFO:teuthology.provision.fog.smithi115:Node is ready
2021-02-15T21:48:23.507 INFO:teuthology.orchestra.run.smithi115.stdout:smithi115.front.sepia.ceph.com
2021-02-15T21:48:23.564 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.15.115 smithi115.front.sepia.ceph.com smithi115
2021-02-15T21:48:23.896 INFO:teuthology.provision.fog.smithi115:Deploy complete!
2021-02-15T21:48:26.244 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-02-15T21:48:26.576 INFO:teuthology.provision.fog.smithi150:Node is ready
2021-02-15T21:48:26.596 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com
2021-02-15T21:48:26.664 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150
2021-02-15T21:48:27.031 INFO:teuthology.provision.fog.smithi150:Deploy complete!
2021-02-15T21:48:27.222 INFO:teuthology.lock.ops:Checking smithi115.front.sepia.ceph.com
2021-02-15T21:48:27.222 INFO:teuthology.lock.ops:New key found. Updating...
2021-02-15T21:48:27.239 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com
2021-02-15T21:48:27.240 INFO:teuthology.lock.ops:New key found. Updating...
2021-02-15T21:48:27.255 INFO:teuthology.lock.ops:Updating [smithi115.front.sepia.ceph.com]: set os type and version on server
2021-02-15T21:48:27.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-15T21:48:27.535 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64
2021-02-15T21:48:27.589 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Linux"
2021-02-15T21:48:27.589 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="8 (Core)"
2021-02-15T21:48:27.589 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="8"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el8"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://www.centos.org/"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-02-15T21:48:27.590 INFO:teuthology.orchestra.run.smithi115.stdout:
2021-02-15T21:48:27.591 INFO:teuthology.orchestra.run.smithi115.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-02-15T21:48:27.591 INFO:teuthology.orchestra.run.smithi115.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-02-15T21:48:27.591 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-02-15T21:48:27.591 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-02-15T21:48:27.591 INFO:teuthology.orchestra.run.smithi115.stdout:
2021-02-15T21:48:27.593 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server
2021-02-15T21:48:27.620 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server
2021-02-15T21:48:27.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-15T21:48:27.890 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Linux"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8 (Core)"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Linux 8 (Core)"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2021-02-15T21:48:27.944 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.centos.org/"
2021-02-15T21:48:27.945 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugs.centos.org/"
2021-02-15T21:48:27.945 INFO:teuthology.orchestra.run.smithi150.stdout:
2021-02-15T21:48:27.945 INFO:teuthology.orchestra.run.smithi150.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8"
2021-02-15T21:48:27.945 INFO:teuthology.orchestra.run.smithi150.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8"
2021-02-15T21:48:27.945 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="centos"
2021-02-15T21:48:27.945 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8"
2021-02-15T21:48:27.945 INFO:teuthology.orchestra.run.smithi150.stdout:
2021-02-15T21:48:27.946 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server
2021-02-15T21:48:28.121 INFO:teuthology.dispatcher.supervisor:Running job 5885325
2021-02-15T21:48:28.121 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/virtualenv/bin/teuthology -v --owner scheduled_rzarzynski@teuthology --archive /home/teuthworker/archive/rzarzynski-2021-02-15_21:09:02-rados-master-distro-basic-smithi/5885325 --name rzarzynski-2021-02-15_21:09:02-rados-master-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} -- /home/teuthworker/archive/rzarzynski-2021-02-15_21:09:02-rados-master-distro-basic-smithi/5885325/orig.config.yaml
2021-02-15T21:48:28.125 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rzarzynski-2021-02-15_21:09:02-rados-master-distro-basic-smithi/5885325
2021-02-15T21:48:28.126 INFO:teuthology.dispatcher.supervisor:Job PID: 19773
2021-02-15T21:48:28.126 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-02-16T09:48:43.609 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-02-16T09:48:43.864 INFO:teuthology.kill:Killing Pids: {19773}
2021-02-16T09:48:44.842 INFO:teuthology.kill:Nuking machines: ['smithi115', 'smithi150']
2021-02-16T09:48:45.631 INFO:teuthology.kill:2021-02-16 09:48:45,630.630 INFO:teuthology.nuke:targets:
2021-02-16T09:48:45.631 INFO:teuthology.kill:  smithi115.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy1+xiU/uSm+9DUB6f1S2fA5ivAA5CKm0oCwSYTcLV5bwwZvEVBW7C71AzSNy1eXB+jPnUPlAHm3xM5gwnWVLRm8QHPxKVgQ/ABshE27N2GetEL+JT0NVasbo8UP+J70u8FGGrgyPNVKNNFfcByc6wGSRjSHRWnJwEmhGmO0ochbGd6fQedzXG3OGpGtYOsLhWMk6htBm8JQhcOayRYBgK7DHycSdpJ1jFAwbnqukTzL7+OviBmnV0m55b5+rG2qkygRkLVSGzuOg06CQaq0ujsCjXce3WDM/Icq0VwKjWH37XfIzMjnrLJAzY6a/D2q9GokUF6slWlk66Ud0SdAFaPR9lwigT3vZSPeyHWauEDIQGd0EvPyJ5/jF6tBsZfT/eyr7Q9FF8Rol92QcBeCbFC4BSyLisDZyjaXXlzuyj2QyD846Zop9rpx1h/uXSV+EFzS617E0w5r4FiAttNLn8foEzqqdVLgHilvnsd5gC+UGu9MvhsTswwI6zYfWApks=
2021-02-16T09:48:45.631 INFO:teuthology.kill:  smithi150.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjFJMaicEN2XAUzoZos78CKg/JZnnb1xrT1iWLqav4RnE+ikKPmrz7DE5chHn3k41cDdhlIRGBV61DU0DUNZmJaQBUJayD/yzieJOQRXnYT0TgJc29xhzACL4NSIDL9KAqX48IMqH+8wNiIJV4ACArgpU6KloxNNVQDMuCjFrdISFmyZTkcOoxCUBMdyFG73Wm8HbeQ6aXWZvNyPZucaSMSVkLarCXdN/D5tgjOJuytjhWrMVqFJH18moNr40r0w7i7KprLtp/Hzk1zJ8WxJYBsNHq7QBX4IDLzl817F9Em12WlCnkGq09uRBDyjhSI9WMvgVsYxHoHm39jqYfYGfc+wk5aTqJ/Leu+FrIH2T8e+Yohq+tR5EwohP4XgPS7eag/LrKRFavQX1JGcCWMVVSpYFH45i6jEsMyD+LRUQWxiDaaZN+2rreGftZuF1RZxIjd91qzdH1zApWgQBbwUicA3SphPolWouREu8R5gBL1o1h3/E3BB/+7xUt/RQlgr8=
2021-02-16T09:48:45.631 INFO:teuthology.kill:2021-02-16 09:48:45,631.631 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-16T09:48:45.637 INFO:teuthology.kill:2021-02-16 09:48:45,637.637 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-16T09:48:45.695 INFO:teuthology.kill:2021-02-16 09:48:45,695.695 INFO:teuthology.orchestra.console:Power off smithi150
2021-02-16T09:48:45.708 INFO:teuthology.kill:2021-02-16 09:48:45,707.707 INFO:teuthology.orchestra.console:Power off smithi115
2021-02-16T09:48:54.063 INFO:teuthology.kill:2021-02-16 09:48:54,063.063 INFO:teuthology.orchestra.console:Power off for smithi150 completed
2021-02-16T09:48:54.073 INFO:teuthology.kill:2021-02-16 09:48:54,073.073 INFO:teuthology.orchestra.console:Power off for smithi115 completed
2021-02-16T09:48:54.295 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-02-16T09:48:54.295 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-02-16T09:48:54.295 INFO:teuthology.misc:Compressing logs...
2021-02-16T09:48:54.296 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-16T09:49:54.299 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/misc.py", line 1372, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-02-16T09:49:54.359 INFO:teuthology.kill:No teuthology processes running
2021-02-16T09:49:55.235 INFO:teuthology.kill:Nuking machines: ['smithi115', 'smithi150']
2021-02-16T09:49:56.006 INFO:teuthology.kill:2021-02-16 09:49:56,005.005 INFO:teuthology.nuke:targets:
2021-02-16T09:49:56.006 INFO:teuthology.kill:  smithi115.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy1+xiU/uSm+9DUB6f1S2fA5ivAA5CKm0oCwSYTcLV5bwwZvEVBW7C71AzSNy1eXB+jPnUPlAHm3xM5gwnWVLRm8QHPxKVgQ/ABshE27N2GetEL+JT0NVasbo8UP+J70u8FGGrgyPNVKNNFfcByc6wGSRjSHRWnJwEmhGmO0ochbGd6fQedzXG3OGpGtYOsLhWMk6htBm8JQhcOayRYBgK7DHycSdpJ1jFAwbnqukTzL7+OviBmnV0m55b5+rG2qkygRkLVSGzuOg06CQaq0ujsCjXce3WDM/Icq0VwKjWH37XfIzMjnrLJAzY6a/D2q9GokUF6slWlk66Ud0SdAFaPR9lwigT3vZSPeyHWauEDIQGd0EvPyJ5/jF6tBsZfT/eyr7Q9FF8Rol92QcBeCbFC4BSyLisDZyjaXXlzuyj2QyD846Zop9rpx1h/uXSV+EFzS617E0w5r4FiAttNLn8foEzqqdVLgHilvnsd5gC+UGu9MvhsTswwI6zYfWApks=
2021-02-16T09:49:56.006 INFO:teuthology.kill:  smithi150.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjFJMaicEN2XAUzoZos78CKg/JZnnb1xrT1iWLqav4RnE+ikKPmrz7DE5chHn3k41cDdhlIRGBV61DU0DUNZmJaQBUJayD/yzieJOQRXnYT0TgJc29xhzACL4NSIDL9KAqX48IMqH+8wNiIJV4ACArgpU6KloxNNVQDMuCjFrdISFmyZTkcOoxCUBMdyFG73Wm8HbeQ6aXWZvNyPZucaSMSVkLarCXdN/D5tgjOJuytjhWrMVqFJH18moNr40r0w7i7KprLtp/Hzk1zJ8WxJYBsNHq7QBX4IDLzl817F9Em12WlCnkGq09uRBDyjhSI9WMvgVsYxHoHm39jqYfYGfc+wk5aTqJ/Leu+FrIH2T8e+Yohq+tR5EwohP4XgPS7eag/LrKRFavQX1JGcCWMVVSpYFH45i6jEsMyD+LRUQWxiDaaZN+2rreGftZuF1RZxIjd91qzdH1zApWgQBbwUicA3SphPolWouREu8R5gBL1o1h3/E3BB/+7xUt/RQlgr8=
2021-02-16T09:49:56.028 INFO:teuthology.kill:2021-02-16 09:49:56,028.028 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-16T09:49:56.031 INFO:teuthology.kill:2021-02-16 09:49:56,031.031 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-16T09:49:56.084 INFO:teuthology.kill:2021-02-16 09:49:56,084.084 INFO:teuthology.orchestra.console:Power off smithi150
2021-02-16T09:49:56.098 INFO:teuthology.kill:2021-02-16 09:49:56,098.098 INFO:teuthology.orchestra.console:Power off smithi115
2021-02-16T09:49:56.228 INFO:teuthology.kill:2021-02-16 09:49:56,228.228 INFO:teuthology.orchestra.console:Power off for smithi150 completed
2021-02-16T09:49:56.240 INFO:teuthology.kill:2021-02-16 09:49:56,240.240 INFO:teuthology.orchestra.console:Power off for smithi115 completed
2021-02-16T09:49:56.371 INFO:teuthology.kill:2021-02-16 09:49:56,370.370 INFO:teuthology.lock.ops:unlocked smithi150.front.sepia.ceph.com
2021-02-16T09:49:56.379 INFO:teuthology.kill:2021-02-16 09:49:56,379.379 INFO:teuthology.lock.ops:unlocked smithi115.front.sepia.ceph.com
2021-02-16T09:51:56.548 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-02-16T09:51:56.638 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-02-16T09:51:56.638 INFO:teuthology.nuke:Checking targets against current locks
2021-02-16T09:51:56.716 INFO:teuthology.nuke:Not nuking smithi115.front.sepia.ceph.com because description doesn't match
2021-02-16T09:51:56.716 INFO:teuthology.nuke:Not nuking smithi150.front.sepia.ceph.com because description doesn't match