2021-03-03T19:21:11.020 INFO:root:teuthology version: 1.1.0-3082387b
2021-03-03T19:21:11.144 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging
2021-03-03T19:21:11.146 INFO:teuthology.lock.ops:Updating [ubuntu@smithi106.front.sepia.ceph.com]: reset os type and version on server
2021-03-03T19:21:11.146 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2021-03-03T19:21:11.164 INFO:teuthology.lock.ops:Node 'ubuntu@smithi106.front.sepia.ceph.com' reimaging is complete
2021-03-03T19:21:11.165 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi205.front.sepia.ceph.com' reimaging
2021-03-03T19:21:11.165 INFO:teuthology.lock.ops:Updating [ubuntu@smithi205.front.sepia.ceph.com]: reset os type and version on server
2021-03-03T19:21:11.165 INFO:teuthology.lock.ops:Updating smithi205.front.sepia.ceph.com on lock server
2021-03-03T19:21:11.193 INFO:teuthology.lock.ops:Node 'ubuntu@smithi205.front.sepia.ceph.com' reimaging is complete
2021-03-03T19:21:11.396 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of rhel 8.3
2021-03-03T19:21:11.408 INFO:teuthology.provision.fog.smithi205:Scheduling deploy of rhel 8.3
2021-03-03T19:21:11.800 INFO:teuthology.orchestra.console:Power off smithi205
2021-03-03T19:21:11.820 INFO:teuthology.orchestra.console:Power off smithi106
2021-03-03T19:21:20.197 INFO:teuthology.orchestra.console:Power off for smithi205 completed
2021-03-03T19:21:20.298 INFO:teuthology.orchestra.console:Power on smithi205
2021-03-03T19:21:24.319 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2021-03-03T19:21:24.421 INFO:teuthology.orchestra.console:Power on smithi106
2021-03-03T19:21:28.677 INFO:teuthology.orchestra.console:Power on for smithi205 completed
2021-03-03T19:21:28.690 INFO:teuthology.orchestra.console:Power on for smithi106 completed
2021-03-03T19:21:28.778 INFO:teuthology.provision.fog.smithi205:Waiting for deploy to finish
2021-03-03T19:21:28.791 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish
2021-03-03T19:25:26.116 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-03-03T19:25:46.740 INFO:teuthology.provision.fog.smithi205:Node is ready
2021-03-03T19:25:46.760 INFO:teuthology.orchestra.run.smithi205.stdout:smithi205.front.sepia.ceph.com
2021-03-03T19:25:46.817 INFO:teuthology.orchestra.run.smithi205.stdout:172.21.15.205 smithi205.front.sepia.ceph.com smithi205
2021-03-03T19:25:47.172 INFO:teuthology.provision.fog.smithi205:Deploy complete!
2021-03-03T19:26:15.942 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-03-03T19:26:16.257 INFO:teuthology.provision.fog.smithi106:Node is ready
2021-03-03T19:26:16.272 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com
2021-03-03T19:26:16.327 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106
2021-03-03T19:26:16.669 INFO:teuthology.provision.fog.smithi106:Deploy complete!
2021-03-03T19:26:17.000 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com
2021-03-03T19:26:17.072 INFO:teuthology.lock.ops:New key found. Updating...
2021-03-03T19:26:17.091 INFO:teuthology.lock.ops:Checking smithi205.front.sepia.ceph.com
2021-03-03T19:26:17.091 INFO:teuthology.lock.ops:New key found. Updating...
2021-03-03T19:26:17.108 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server
2021-03-03T19:26:17.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-03T19:26:17.433 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64
2021-03-03T19:26:17.488 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Red Hat Enterprise Linux"
2021-03-03T19:26:17.488 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8.3 (Ootpa)"
2021-03-03T19:26:17.489 INFO:teuthology.orchestra.run.smithi106.stdout:ID="rhel"
2021-03-03T19:26:17.489 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="fedora"
2021-03-03T19:26:17.489 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8.3"
2021-03-03T19:26:17.490 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8"
2021-03-03T19:26:17.490 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-03-03T19:26:17.490 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31"
2021-03-03T19:26:17.490 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-03-03T19:26:17.491 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.redhat.com/"
2021-03-03T19:26:17.491 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-03-03T19:26:17.491 INFO:teuthology.orchestra.run.smithi106.stdout:
2021-03-03T19:26:17.491 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-03-03T19:26:17.492 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-03-03T19:26:17.493 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-03-03T19:26:17.493 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-03-03T19:26:17.495 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2021-03-03T19:26:17.511 INFO:teuthology.lock.ops:Updating [smithi205.front.sepia.ceph.com]: set os type and version on server
2021-03-03T19:26:17.512 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-03T19:26:17.840 INFO:teuthology.orchestra.run.smithi205.stdout:x86_64
2021-03-03T19:26:17.899 INFO:teuthology.orchestra.run.smithi205.stdout:NAME="Red Hat Enterprise Linux"
2021-03-03T19:26:17.899 INFO:teuthology.orchestra.run.smithi205.stdout:VERSION="8.3 (Ootpa)"
2021-03-03T19:26:17.900 INFO:teuthology.orchestra.run.smithi205.stdout:ID="rhel"
2021-03-03T19:26:17.900 INFO:teuthology.orchestra.run.smithi205.stdout:ID_LIKE="fedora"
2021-03-03T19:26:17.900 INFO:teuthology.orchestra.run.smithi205.stdout:VERSION_ID="8.3"
2021-03-03T19:26:17.900 INFO:teuthology.orchestra.run.smithi205.stdout:PLATFORM_ID="platform:el8"
2021-03-03T19:26:17.901 INFO:teuthology.orchestra.run.smithi205.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-03-03T19:26:17.901 INFO:teuthology.orchestra.run.smithi205.stdout:ANSI_COLOR="0;31"
2021-03-03T19:26:17.901 INFO:teuthology.orchestra.run.smithi205.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-03-03T19:26:17.901 INFO:teuthology.orchestra.run.smithi205.stdout:HOME_URL="https://www.redhat.com/"
2021-03-03T19:26:17.902 INFO:teuthology.orchestra.run.smithi205.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-03-03T19:26:17.902 INFO:teuthology.orchestra.run.smithi205.stdout:
2021-03-03T19:26:17.902 INFO:teuthology.orchestra.run.smithi205.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-03-03T19:26:17.903 INFO:teuthology.orchestra.run.smithi205.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-03-03T19:26:17.903 INFO:teuthology.orchestra.run.smithi205.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-03-03T19:26:17.903 INFO:teuthology.orchestra.run.smithi205.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-03-03T19:26:17.904 INFO:teuthology.lock.ops:Updating smithi205.front.sepia.ceph.com on lock server
2021-03-03T19:26:17.993 INFO:teuthology.dispatcher.supervisor:Running job 5929973
2021-03-03T19:26:17.994 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology -v --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2021-03-03_16:41:22-rados-wip-sage2-testing-2021-03-03-0744-pacific-distro-basic-smithi/5929973 --name sage-2021-03-03_16:41:22-rados-wip-sage2-testing-2021-03-03-0744-pacific-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects} -- /home/teuthworker/archive/sage-2021-03-03_16:41:22-rados-wip-sage2-testing-2021-03-03-0744-pacific-distro-basic-smithi/5929973/orig.config.yaml
2021-03-03T19:26:18.020 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/sage-2021-03-03_16:41:22-rados-wip-sage2-testing-2021-03-03-0744-pacific-distro-basic-smithi/5929973
2021-03-03T19:26:18.020 INFO:teuthology.dispatcher.supervisor:Job PID: 11946
2021-03-03T19:26:18.021 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-03-04T07:27:07.483 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-03-04T07:27:07.786 INFO:teuthology.kill:Killing Pids: {11946}
2021-03-04T07:27:09.339 INFO:teuthology.kill:Nuking machines: ['smithi106', 'smithi205']
2021-03-04T07:27:10.304 INFO:teuthology.kill:2021-03-04 07:27:10,303.303 INFO:teuthology.nuke:targets:
2021-03-04T07:27:10.305 INFO:teuthology.kill:  smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtxL3nTIJ0ShQXMvLwdGQ/A1CXuj8KCECNfxeiYljg8oO2sNl5ceJFVRP1nCu7MM98ZRwq51UgHGCK6BRLBNQRUWD9xyHDpZAdFsFT4rmP28GBWCvUa1Bno7Lc+CeOMm1gJPgUVnkQOAgXeVWk2qCq9enc56Lq2ISajW/CfLl+bahvPkiRnncqm36YNfpUHoizUROLSBtzHog4j+U44oXv7S+lG508Jfji05sXRtrT9vPUVrp4S/G/zip3Wcw012U1ojWr9yubE1ZEYzt8IOSWbt3vjvsHy5fS93pvTlmrx4q5Foh2P7uO+7KAGtyqHFXJeLjC90AJS+1vSqwK7wcakFZQUvzE0DRsasB2qCqiHVrSaJlLapz1i701imxUoprkSLcCoHZqvHvPMRomX8r/tShbtMSHNV03nTzNLFKVHSk6eVXCchTSkzYPjFgc5l82ZvFH+B3atjyyD6NNZe/LChDN1C5klF+D6poaEZDFIjKMkhfvCmXVC1SK7LFwSiU=
2021-03-04T07:27:10.305 INFO:teuthology.kill:  smithi205.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDinel/ZIjkr931PA/Ba1g2KlocfZFA3WSkRqnMc292Q9v2ixHtkPj62O2QCpTQAKwEvknPJdSS0tOognTg5Y4n+diXIW5dB3C/R6wNTkSsSXCKOf4IBJfJbjHbOHB4PzAf/pDJtO0HNYPcrCUx1BLVR/v9Ue7pTh+zDRqN2HkTeC1gnWMN6w7A/1igZB582LK2Ymst6dr46fzbZ6mh+aS+AzAquoLALtHSrDtGzp5ArIInybOqnFk30esBvaXUf0dj81s2m9/Um/J+TK8j+w7T6EsOxsu5Pv2/mCZlfQEQvlX57CocOVCnqwKnrqDeW7awJ23WRxrYvldzeQnwRRnfZeW+0Rq8DdbJOvbvgPpSY9p4+6kxFPdT4ifW+6OdnfSRiv1hre4bUVY9RALnUkQzd1VciLnWmCT5UdaXkVthucExWSI1yiT01b8oWHybJnQsxtpRkrcJhH4PnHpGyMsWTgisf2LggxImljq3gX/obzPxQDmdWJCaiJz6W7t+J50=
2021-03-04T07:27:10.305 INFO:teuthology.kill:2021-03-04 07:27:10,304.304 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-04T07:27:10.306 INFO:teuthology.kill:2021-03-04 07:27:10,306.306 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-04T07:27:10.369 INFO:teuthology.kill:2021-03-04 07:27:10,368.368 INFO:teuthology.orchestra.console:Power off smithi205
2021-03-04T07:27:10.381 INFO:teuthology.kill:2021-03-04 07:27:10,381.381 INFO:teuthology.orchestra.console:Power off smithi106
2021-03-04T07:27:18.740 INFO:teuthology.kill:2021-03-04 07:27:18,740.740 INFO:teuthology.orchestra.console:Power off for smithi205 completed
2021-03-04T07:27:18.745 INFO:teuthology.kill:2021-03-04 07:27:18,745.745 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2021-03-04T07:27:18.965 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-03-04T07:27:18.966 INFO:teuthology.task.internal:roles: ubuntu@smithi205.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-03-04T07:27:18.966 INFO:teuthology.misc:Compressing logs...
2021-03-04T07:27:18.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-04T07:28:19.031 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/misc.py", line 1374, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-03-04T07:28:19.089 INFO:teuthology.kill:No teuthology processes running
2021-03-04T07:28:20.839 INFO:teuthology.kill:Nuking machines: ['smithi106', 'smithi205']
2021-03-04T07:28:21.842 INFO:teuthology.kill:2021-03-04 07:28:21,842.842 INFO:teuthology.nuke:targets:
2021-03-04T07:28:21.843 INFO:teuthology.kill:  smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtxL3nTIJ0ShQXMvLwdGQ/A1CXuj8KCECNfxeiYljg8oO2sNl5ceJFVRP1nCu7MM98ZRwq51UgHGCK6BRLBNQRUWD9xyHDpZAdFsFT4rmP28GBWCvUa1Bno7Lc+CeOMm1gJPgUVnkQOAgXeVWk2qCq9enc56Lq2ISajW/CfLl+bahvPkiRnncqm36YNfpUHoizUROLSBtzHog4j+U44oXv7S+lG508Jfji05sXRtrT9vPUVrp4S/G/zip3Wcw012U1ojWr9yubE1ZEYzt8IOSWbt3vjvsHy5fS93pvTlmrx4q5Foh2P7uO+7KAGtyqHFXJeLjC90AJS+1vSqwK7wcakFZQUvzE0DRsasB2qCqiHVrSaJlLapz1i701imxUoprkSLcCoHZqvHvPMRomX8r/tShbtMSHNV03nTzNLFKVHSk6eVXCchTSkzYPjFgc5l82ZvFH+B3atjyyD6NNZe/LChDN1C5klF+D6poaEZDFIjKMkhfvCmXVC1SK7LFwSiU=
2021-03-04T07:28:21.844 INFO:teuthology.kill:  smithi205.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDinel/ZIjkr931PA/Ba1g2KlocfZFA3WSkRqnMc292Q9v2ixHtkPj62O2QCpTQAKwEvknPJdSS0tOognTg5Y4n+diXIW5dB3C/R6wNTkSsSXCKOf4IBJfJbjHbOHB4PzAf/pDJtO0HNYPcrCUx1BLVR/v9Ue7pTh+zDRqN2HkTeC1gnWMN6w7A/1igZB582LK2Ymst6dr46fzbZ6mh+aS+AzAquoLALtHSrDtGzp5ArIInybOqnFk30esBvaXUf0dj81s2m9/Um/J+TK8j+w7T6EsOxsu5Pv2/mCZlfQEQvlX57CocOVCnqwKnrqDeW7awJ23WRxrYvldzeQnwRRnfZeW+0Rq8DdbJOvbvgPpSY9p4+6kxFPdT4ifW+6OdnfSRiv1hre4bUVY9RALnUkQzd1VciLnWmCT5UdaXkVthucExWSI1yiT01b8oWHybJnQsxtpRkrcJhH4PnHpGyMsWTgisf2LggxImljq3gX/obzPxQDmdWJCaiJz6W7t+J50=
2021-03-04T07:28:21.862 INFO:teuthology.kill:2021-03-04 07:28:21,862.862 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-04T07:28:21.865 INFO:teuthology.kill:2021-03-04 07:28:21,865.865 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-04T07:28:21.925 INFO:teuthology.kill:2021-03-04 07:28:21,924.924 INFO:teuthology.orchestra.console:Power off smithi205
2021-03-04T07:28:21.941 INFO:teuthology.kill:2021-03-04 07:28:21,940.940 INFO:teuthology.orchestra.console:Power off smithi106
2021-03-04T07:28:22.089 INFO:teuthology.kill:2021-03-04 07:28:22,089.089 INFO:teuthology.orchestra.console:Power off for smithi205 completed
2021-03-04T07:28:22.095 INFO:teuthology.kill:2021-03-04 07:28:22,095.095 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2021-03-04T07:28:22.231 INFO:teuthology.kill:2021-03-04 07:28:22,230.230 INFO:teuthology.lock.ops:unlocked smithi205.front.sepia.ceph.com
2021-03-04T07:28:22.234 INFO:teuthology.kill:2021-03-04 07:28:22,234.234 INFO:teuthology.lock.ops:unlocked smithi106.front.sepia.ceph.com
2021-03-04T07:30:22.518 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-03-04T07:30:22.618 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-03-04T07:30:22.619 INFO:teuthology.nuke:Checking targets against current locks
2021-03-04T07:30:22.696 INFO:teuthology.nuke:Not nuking smithi205.front.sepia.ceph.com because description doesn't match
2021-03-04T07:30:22.697 INFO:teuthology.nuke:Not nuking smithi106.front.sepia.ceph.com because description doesn't match