2021-02-25T01:14:48.730 INFO:root:teuthology version: 1.1.0-9a6b849c
2021-02-25T01:14:48.848 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi091.front.sepia.ceph.com' reimaging
2021-02-25T01:14:48.849 INFO:teuthology.lock.ops:Updating [ubuntu@smithi091.front.sepia.ceph.com]: reset os type and version on server
2021-02-25T01:14:48.850 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server
2021-02-25T01:14:48.865 INFO:teuthology.lock.ops:Node 'ubuntu@smithi091.front.sepia.ceph.com' reimaging is complete
2021-02-25T01:14:48.865 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi148.front.sepia.ceph.com' reimaging
2021-02-25T01:14:48.866 INFO:teuthology.lock.ops:Updating [ubuntu@smithi148.front.sepia.ceph.com]: reset os type and version on server
2021-02-25T01:14:48.866 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server
2021-02-25T01:14:48.884 INFO:teuthology.lock.ops:Node 'ubuntu@smithi148.front.sepia.ceph.com' reimaging is complete
2021-02-25T01:14:49.086 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of rhel 8.3
2021-02-25T01:14:49.093 INFO:teuthology.provision.fog.smithi091:Scheduling deploy of rhel 8.3
2021-02-25T01:14:49.478 INFO:teuthology.orchestra.console:Power off smithi091
2021-02-25T01:14:49.519 INFO:teuthology.orchestra.console:Power off smithi148
2021-02-25T01:14:57.886 INFO:teuthology.orchestra.console:Power off for smithi091 completed
2021-02-25T01:14:57.900 INFO:teuthology.orchestra.console:Power off for smithi148 completed
2021-02-25T01:14:57.988 INFO:teuthology.orchestra.console:Power on smithi091
2021-02-25T01:14:58.001 INFO:teuthology.orchestra.console:Power on smithi148
2021-02-25T01:15:06.368 INFO:teuthology.orchestra.console:Power on for smithi091 completed
2021-02-25T01:15:06.374 INFO:teuthology.orchestra.console:Power on for smithi148 completed
2021-02-25T01:15:06.470 INFO:teuthology.provision.fog.smithi091:Waiting for deploy to finish
2021-02-25T01:15:06.475 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish
2021-02-25T01:19:12.842 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-02-25T01:19:32.929 INFO:teuthology.provision.fog.smithi091:Node is ready
2021-02-25T01:19:32.948 INFO:teuthology.orchestra.run.smithi091.stdout:smithi091.front.sepia.ceph.com
2021-02-25T01:19:33.004 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.15.91 smithi091.front.sepia.ceph.com smithi091
2021-02-25T01:19:33.375 INFO:teuthology.provision.fog.smithi091:Deploy complete!
2021-02-25T01:20:02.565 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-02-25T01:20:02.892 INFO:teuthology.provision.fog.smithi148:Node is ready
2021-02-25T01:20:02.910 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148.front.sepia.ceph.com
2021-02-25T01:20:02.971 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148
2021-02-25T01:20:03.317 INFO:teuthology.provision.fog.smithi148:Deploy complete!
2021-02-25T01:20:03.580 INFO:teuthology.lock.ops:Checking smithi091.front.sepia.ceph.com
2021-02-25T01:20:03.581 INFO:teuthology.lock.ops:New key found. Updating...
2021-02-25T01:20:03.609 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com
2021-02-25T01:20:03.610 INFO:teuthology.lock.ops:New key found. Updating...
2021-02-25T01:20:03.636 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: set os type and version on server
2021-02-25T01:20:03.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-25T01:20:04.007 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64
2021-02-25T01:20:04.076 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="Red Hat Enterprise Linux"
2021-02-25T01:20:04.077 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="8.3 (Ootpa)"
2021-02-25T01:20:04.078 INFO:teuthology.orchestra.run.smithi091.stdout:ID="rhel"
2021-02-25T01:20:04.079 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="fedora"
2021-02-25T01:20:04.080 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="8.3"
2021-02-25T01:20:04.081 INFO:teuthology.orchestra.run.smithi091.stdout:PLATFORM_ID="platform:el8"
2021-02-25T01:20:04.082 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-02-25T01:20:04.083 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31"
2021-02-25T01:20:04.084 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-02-25T01:20:04.087 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.redhat.com/"
2021-02-25T01:20:04.087 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-02-25T01:20:04.088 INFO:teuthology.orchestra.run.smithi091.stdout:
2021-02-25T01:20:04.089 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-02-25T01:20:04.090 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-02-25T01:20:04.091 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-02-25T01:20:04.091 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-02-25T01:20:04.103 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server
2021-02-25T01:20:04.128 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server
2021-02-25T01:20:04.128 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-25T01:20:04.393 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64
2021-02-25T01:20:04.447 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Red Hat Enterprise Linux"
2021-02-25T01:20:04.447 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8.3 (Ootpa)"
2021-02-25T01:20:04.448 INFO:teuthology.orchestra.run.smithi148.stdout:ID="rhel"
2021-02-25T01:20:04.448 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="fedora"
2021-02-25T01:20:04.448 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8.3"
2021-02-25T01:20:04.448 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8"
2021-02-25T01:20:04.448 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-02-25T01:20:04.449 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31"
2021-02-25T01:20:04.449 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-02-25T01:20:04.449 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.redhat.com/"
2021-02-25T01:20:04.449 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-02-25T01:20:04.450 INFO:teuthology.orchestra.run.smithi148.stdout:
2021-02-25T01:20:04.450 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-02-25T01:20:04.450 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-02-25T01:20:04.450 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-02-25T01:20:04.450 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-02-25T01:20:04.451 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server
2021-02-25T01:20:04.541 INFO:teuthology.dispatcher.supervisor:Running job 5912526
2021-02-25T01:20:04.541 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-02-24_21:26:56-rados-wip-sage-testing-2021-02-24-1457-distro-basic-smithi/5912526 --name sage-2021-02-24_21:26:56-rados-wip-sage-testing-2021-02-24-1457-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} -- /home/teuthworker/archive/sage-2021-02-24_21:26:56-rados-wip-sage-testing-2021-02-24-1457-distro-basic-smithi/5912526/orig.config.yaml
2021-02-25T01:20:04.555 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-02-24_21:26:56-rados-wip-sage-testing-2021-02-24-1457-distro-basic-smithi/5912526
2021-02-25T01:20:04.556 INFO:teuthology.dispatcher.supervisor:Job PID: 9849
2021-02-25T01:20:04.556 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-02-25T13:20:20.938 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-02-25T13:20:21.283 INFO:teuthology.kill:Killing Pids: {9849}
2021-02-25T13:20:22.316 INFO:teuthology.kill:Nuking machines: ['smithi091', 'smithi148']
2021-02-25T13:20:23.255 INFO:teuthology.kill:2021-02-25 13:20:23,254.254 INFO:teuthology.nuke:targets:
2021-02-25T13:20:23.255 INFO:teuthology.kill:  smithi091.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIZqezdZlti9nOLhKHYQ7fKXg8Filq2SzGotRkguZp22yNgg9rWC0aRm+zM5vhswWuGNEg6wREkAEDQv1f6XC0sLaFxGYiozOQ9/HCC1XEXpD7ul08Qm1OKalDz7ZUlkwi0Epu1p8enIB6BrnzOr0C72c7MG9nLXiZDSCvAdZUu5TH1OFSTX2PL6iCpKBc+9+my9iZJKSHJKQnA6pFG7LwJ0nTc9rw2UoeHdr2B7cOHr547kBpVy6w0oUOvZWALdiYWEsL0MYjRdvvqu8gpuIZPCUUzo/OwSKsr87ETbcJHkgztO0VW+GYucYM7oqBF+bANdFN1mTSa4pzzRLM5W9Lm2/HHTfr2C9lwIdcwYNga1HSZiBnM01O4c65jNslPqspXxPxSRTETrls15q09Z/dneUkTPmvTN0fyrved7L2o7m5kt9Aj5XSeaTHVNxfs7ruQnr/XXl3facbaPVCV5MpbDHyHAkaFtEqAjPEk1lX7ISL4Y/ExZlREIFrWXuSSCE=
2021-02-25T13:20:23.256 INFO:teuthology.kill:  smithi148.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6tjaGjGSB9fOeLViwIp1PYIO3N1zzHhrV7ND0eOZuc3RDx4bi7HtjjI20uAxFV8N16YWxIHoBzvQbavO1evj4JJRzSd4GO0RG9St9cQOhn4yDafeZymRr9VN+E4BtYuHCxeLyRk3JcBaFBIw/IYHOu3FYxvkoq4HlzFp1gH3Csb95Y5n8Z/VksuVLE79Blg0PFFLH5XXF2E+m8V4RUiYM8NSGc6QYoUs3lstgtf5kcWQejA1XDuQw8dsDXSkDhsQc6Cwe6m14BGFpThgabMCtPyzpurPj9up++dejKTEgZPdj5aSg81d14z7U5rZewRyzSDLdRXZz2tLOgr/bJItQ51iBKPbBhYNgz5+rtteEwS5JJERamrU0n7Hyca42N7cxda6KdriJy8RpuPniS8aRsUzWxGqs77VBprQaU4orcXqs3v1Pldas0NqLAmUwLxekqgBCLEPLkmFeEyS1SESL/Yub/4dSj792ZUyllZWOv5WOCd4MzsqqGmOimqEppbs=
2021-02-25T13:20:23.256 INFO:teuthology.kill:2021-02-25 13:20:23,255.255 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-25T13:20:23.261 INFO:teuthology.kill:2021-02-25 13:20:23,261.261 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-25T13:20:23.323 INFO:teuthology.kill:2021-02-25 13:20:23,323.323 INFO:teuthology.orchestra.console:Power off smithi091
2021-02-25T13:20:23.339 INFO:teuthology.kill:2021-02-25 13:20:23,338.338 INFO:teuthology.orchestra.console:Power off smithi148
2021-02-25T13:20:31.717 INFO:teuthology.kill:2021-02-25 13:20:31,717.717 INFO:teuthology.orchestra.console:Power off for smithi148 completed
2021-02-25T13:20:35.821 INFO:teuthology.kill:2021-02-25 13:20:35,821.821 INFO:teuthology.orchestra.console:Power off for smithi091 completed
2021-02-25T13:20:36.056 INFO:teuthology.task.internal:roles: ubuntu@smithi091.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-02-25T13:20:36.057 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-02-25T13:20:36.057 INFO:teuthology.misc:Compressing logs...
2021-02-25T13:20:36.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-02-25T13:20:54.466 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/misc.py", line 1374, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9a6b849c7975e1b07e5495aaa5b961c0d22cad1c/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-02-25T13:20:54.523 INFO:teuthology.kill:No teuthology processes running
2021-02-25T13:20:55.482 INFO:teuthology.kill:Nuking machines: ['smithi091', 'smithi148']
2021-02-25T13:20:56.274 INFO:teuthology.kill:2021-02-25 13:20:56,274.274 INFO:teuthology.nuke:targets:
2021-02-25T13:20:56.275 INFO:teuthology.kill:  smithi091.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIZqezdZlti9nOLhKHYQ7fKXg8Filq2SzGotRkguZp22yNgg9rWC0aRm+zM5vhswWuGNEg6wREkAEDQv1f6XC0sLaFxGYiozOQ9/HCC1XEXpD7ul08Qm1OKalDz7ZUlkwi0Epu1p8enIB6BrnzOr0C72c7MG9nLXiZDSCvAdZUu5TH1OFSTX2PL6iCpKBc+9+my9iZJKSHJKQnA6pFG7LwJ0nTc9rw2UoeHdr2B7cOHr547kBpVy6w0oUOvZWALdiYWEsL0MYjRdvvqu8gpuIZPCUUzo/OwSKsr87ETbcJHkgztO0VW+GYucYM7oqBF+bANdFN1mTSa4pzzRLM5W9Lm2/HHTfr2C9lwIdcwYNga1HSZiBnM01O4c65jNslPqspXxPxSRTETrls15q09Z/dneUkTPmvTN0fyrved7L2o7m5kt9Aj5XSeaTHVNxfs7ruQnr/XXl3facbaPVCV5MpbDHyHAkaFtEqAjPEk1lX7ISL4Y/ExZlREIFrWXuSSCE=
2021-02-25T13:20:56.275 INFO:teuthology.kill:  smithi148.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6tjaGjGSB9fOeLViwIp1PYIO3N1zzHhrV7ND0eOZuc3RDx4bi7HtjjI20uAxFV8N16YWxIHoBzvQbavO1evj4JJRzSd4GO0RG9St9cQOhn4yDafeZymRr9VN+E4BtYuHCxeLyRk3JcBaFBIw/IYHOu3FYxvkoq4HlzFp1gH3Csb95Y5n8Z/VksuVLE79Blg0PFFLH5XXF2E+m8V4RUiYM8NSGc6QYoUs3lstgtf5kcWQejA1XDuQw8dsDXSkDhsQc6Cwe6m14BGFpThgabMCtPyzpurPj9up++dejKTEgZPdj5aSg81d14z7U5rZewRyzSDLdRXZz2tLOgr/bJItQ51iBKPbBhYNgz5+rtteEwS5JJERamrU0n7Hyca42N7cxda6KdriJy8RpuPniS8aRsUzWxGqs77VBprQaU4orcXqs3v1Pldas0NqLAmUwLxekqgBCLEPLkmFeEyS1SESL/Yub/4dSj792ZUyllZWOv5WOCd4MzsqqGmOimqEppbs=
2021-02-25T13:20:56.297 INFO:teuthology.kill:2021-02-25 13:20:56,297.297 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-25T13:20:56.300 INFO:teuthology.kill:2021-02-25 13:20:56,300.300 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-02-25T13:20:56.357 INFO:teuthology.kill:2021-02-25 13:20:56,357.357 INFO:teuthology.orchestra.console:Power off smithi148
2021-02-25T13:20:56.370 INFO:teuthology.kill:2021-02-25 13:20:56,370.370 INFO:teuthology.orchestra.console:Power off smithi091
2021-02-25T13:20:56.501 INFO:teuthology.kill:2021-02-25 13:20:56,501.501 INFO:teuthology.orchestra.console:Power off for smithi148 completed
2021-02-25T13:20:56.515 INFO:teuthology.kill:2021-02-25 13:20:56,515.515 INFO:teuthology.orchestra.console:Power off for smithi091 completed
2021-02-25T13:20:56.648 INFO:teuthology.kill:2021-02-25 13:20:56,647.647 INFO:teuthology.lock.ops:unlocked smithi148.front.sepia.ceph.com
2021-02-25T13:20:56.657 INFO:teuthology.kill:2021-02-25 13:20:56,657.657 INFO:teuthology.lock.ops:unlocked smithi091.front.sepia.ceph.com
2021-02-25T13:22:56.856 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-02-25T13:22:57.039 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-02-25T13:22:57.040 INFO:teuthology.nuke:Checking targets against current locks
2021-02-25T13:22:57.112 INFO:teuthology.nuke:Not nuking smithi148.front.sepia.ceph.com because description doesn't match
2021-02-25T13:22:57.112 INFO:teuthology.nuke:Not nuking smithi091.front.sepia.ceph.com because description doesn't match