2021-03-03T01:57:56.702 INFO:root:teuthology version: 1.1.0-3082387b
2021-03-03T01:57:56.795 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi028.front.sepia.ceph.com' reimaging
2021-03-03T01:57:56.796 INFO:teuthology.lock.ops:Updating [ubuntu@smithi028.front.sepia.ceph.com]: reset os type and version on server
2021-03-03T01:57:56.796 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server
2021-03-03T01:57:56.812 INFO:teuthology.lock.ops:Node 'ubuntu@smithi028.front.sepia.ceph.com' reimaging is complete
2021-03-03T01:57:56.812 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi094.front.sepia.ceph.com' reimaging
2021-03-03T01:57:56.813 INFO:teuthology.lock.ops:Updating [ubuntu@smithi094.front.sepia.ceph.com]: reset os type and version on server
2021-03-03T01:57:56.813 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server
2021-03-03T01:57:56.827 INFO:teuthology.lock.ops:Node 'ubuntu@smithi094.front.sepia.ceph.com' reimaging is complete
2021-03-03T01:57:57.038 INFO:teuthology.provision.fog.smithi028:Scheduling deploy of rhel 8.3
2021-03-03T01:57:57.047 INFO:teuthology.provision.fog.smithi094:Scheduling deploy of rhel 8.3
2021-03-03T01:57:57.429 INFO:teuthology.orchestra.console:Power off smithi028
2021-03-03T01:57:57.886 INFO:teuthology.orchestra.console:Power off smithi094
2021-03-03T01:58:05.799 INFO:teuthology.orchestra.console:Power off for smithi028 completed
2021-03-03T01:58:05.900 INFO:teuthology.orchestra.console:Power on smithi028
2021-03-03T01:58:06.251 INFO:teuthology.orchestra.console:Power off for smithi094 completed
2021-03-03T01:58:06.352 INFO:teuthology.orchestra.console:Power on smithi094
2021-03-03T01:58:14.267 INFO:teuthology.orchestra.console:Power on for smithi028 completed
2021-03-03T01:58:14.368 INFO:teuthology.provision.fog.smithi028:Waiting for deploy to finish
2021-03-03T01:58:14.713 INFO:teuthology.orchestra.console:Power on for smithi094 completed
2021-03-03T01:58:14.814 INFO:teuthology.provision.fog.smithi094:Waiting for deploy to finish
2021-03-03T02:02:22.164 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-03-03T02:02:22.481 INFO:teuthology.provision.fog.smithi028:Node is ready
2021-03-03T02:02:22.496 INFO:teuthology.orchestra.run.smithi028.stdout:smithi028.front.sepia.ceph.com
2021-03-03T02:02:22.553 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28 smithi028.front.sepia.ceph.com smithi028
2021-03-03T02:02:22.884 INFO:teuthology.provision.fog.smithi028:Deploy complete!
2021-03-03T02:02:23.314 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-03-03T02:02:44.004 INFO:teuthology.provision.fog.smithi094:Node is ready
2021-03-03T02:02:44.023 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094.front.sepia.ceph.com
2021-03-03T02:02:44.080 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94 smithi094.front.sepia.ceph.com smithi094
2021-03-03T02:02:44.479 INFO:teuthology.provision.fog.smithi094:Deploy complete!
2021-03-03T02:02:44.600 INFO:teuthology.lock.ops:Checking smithi028.front.sepia.ceph.com
2021-03-03T02:02:44.601 INFO:teuthology.lock.ops:New key found. Updating...
2021-03-03T02:02:44.617 INFO:teuthology.lock.ops:Checking smithi094.front.sepia.ceph.com
2021-03-03T02:02:44.618 INFO:teuthology.lock.ops:New key found. Updating...
2021-03-03T02:02:44.632 INFO:teuthology.lock.ops:Updating [smithi028.front.sepia.ceph.com]: set os type and version on server
2021-03-03T02:02:44.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-03T02:02:44.925 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64
2021-03-03T02:02:44.980 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="Red Hat Enterprise Linux"
2021-03-03T02:02:44.980 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="8.3 (Ootpa)"
2021-03-03T02:02:44.981 INFO:teuthology.orchestra.run.smithi028.stdout:ID="rhel"
2021-03-03T02:02:44.981 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="fedora"
2021-03-03T02:02:44.981 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="8.3"
2021-03-03T02:02:44.981 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el8"
2021-03-03T02:02:44.982 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-03-03T02:02:44.982 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31"
2021-03-03T02:02:44.982 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-03-03T02:02:44.982 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://www.redhat.com/"
2021-03-03T02:02:44.983 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-03-03T02:02:44.983 INFO:teuthology.orchestra.run.smithi028.stdout:
2021-03-03T02:02:44.983 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-03-03T02:02:44.983 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-03-03T02:02:44.984 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-03-03T02:02:44.984 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-03-03T02:02:44.985 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server
2021-03-03T02:02:45.002 INFO:teuthology.lock.ops:Updating [smithi094.front.sepia.ceph.com]: set os type and version on server
2021-03-03T02:02:45.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-03T02:02:45.287 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64
2021-03-03T02:02:45.342 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="Red Hat Enterprise Linux"
2021-03-03T02:02:45.342 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="8.3 (Ootpa)"
2021-03-03T02:02:45.343 INFO:teuthology.orchestra.run.smithi094.stdout:ID="rhel"
2021-03-03T02:02:45.343 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="fedora"
2021-03-03T02:02:45.343 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="8.3"
2021-03-03T02:02:45.343 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el8"
2021-03-03T02:02:45.344 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-03-03T02:02:45.344 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31"
2021-03-03T02:02:45.344 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-03-03T02:02:45.345 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://www.redhat.com/"
2021-03-03T02:02:45.345 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-03-03T02:02:45.345 INFO:teuthology.orchestra.run.smithi094.stdout:
2021-03-03T02:02:45.345 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-03-03T02:02:45.346 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-03-03T02:02:45.346 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-03-03T02:02:45.346 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-03-03T02:02:45.347 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server
2021-03-03T02:02:45.427 INFO:teuthology.dispatcher.supervisor:Running job 5927768
2021-03-03T02:02:45.427 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology -v --owner scheduled_rzarzynski@teuthology --archive /home/teuthworker/archive/rzarzynski-2021-03-02_18:30:36-rados-master-distro-basic-smithi/5927768 --name rzarzynski-2021-03-02_18:30:36-rados-master-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/rados_api_tests} -- /home/teuthworker/archive/rzarzynski-2021-03-02_18:30:36-rados-master-distro-basic-smithi/5927768/orig.config.yaml
2021-03-03T02:02:45.430 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rzarzynski-2021-03-02_18:30:36-rados-master-distro-basic-smithi/5927768
2021-03-03T02:02:45.431 INFO:teuthology.dispatcher.supervisor:Job PID: 1804
2021-03-03T02:02:45.431 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-03-03T14:03:37.032 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-03-03T14:03:38.169 INFO:teuthology.kill:Killing Pids: {1804}
2021-03-03T14:03:39.173 INFO:teuthology.kill:Nuking machines: ['smithi028', 'smithi094']
2021-03-03T14:03:40.080 INFO:teuthology.kill:2021-03-03 14:03:40,080.080 INFO:teuthology.nuke:targets:
2021-03-03T14:03:40.081 INFO:teuthology.kill:  smithi028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncUUtrG1RI8lAzaYw0j/mRcmAAEyGH78qKCx8GDgcHHw9eOgnQLmeK4Q5pdIexgDscoc7gwe4lsd1J6XOBDuCF7oDaRClhmEsfyETMHQJAZC5feUBQmieOwbB3LG+sCT2FjaAtVRi3rtW498lRQc+brGoU9v9RQ8+sgzvojf7q090bW170OniLX2W9QxBnUaqPJPFU5ilPURivTdwkY3hxgA2WXbo9amwd3lkJFcAxtNZUItF/ESsFW1X3HJA61pRch7+UjXDkXkxMSCEAmb9yohmJXpXJf9+6kwdtzbwwPlIHx74bifY+4RefothP4Yj/w+l9mNYXi5Ost7ZfmA/gcz7J77tiiPMvhQpkgKW+MNn3EWxGIOEl1bdlYdpATJ0Q110IFk701UA02wMZBhdjeZpPVOIOIdRLUYggBSyzKS+pH88Mevl/xnid7V8GkHv9WYlxOwxJi75+6Q4TygMp0kt8QznFtkm0ONk8ejZ6tehmeSkcQvyhJIkxxR93wU=
2021-03-03T14:03:40.081 INFO:teuthology.kill:  smithi094.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4Mpn0iSw63LihMYHEq8WCkbqDaXxGzdfnr/K11VupCbt1T2BarUrvCJJXp/ew7QNm4cqxEK5L1XMv5yIPQMvhIAut69buY83wfJV8QutM0fTlXBYdNlDPjrxKZ4hNh8B2+h9L+WFi+9tKrT5SLt6dhrmfbUW8FRvXS/QJYoY7ylyJhfg72G+BEXd6Kf7LQclrtyiZ3qbKmcs5f5oDi6AQkmCD44dpCSqsylvpuesAaIVxIh+p/yqzReRuTTDwNwdOXjMMEdVbWHv82v8aiPVD7NyZQmWV9ZPRc1qckz8Krv6vpqid2Sig1dMJNah1PWFI7boj1+IaTk9B3o1tf5jG9ubH6CLigtvEI0go8NFmhnqbrv+d6vvgI02qmmF0qYDETx59HMy7i1UWSUXuUG0g1vkk/J+ozRqa9InURfU58iWXVpoXOvgGVxT6FNiRhYZO8IedcsEgPUfye3QA3bq6BfthGVHNv027MqQw/k7yov8B8TCaqcFLtyrx+qiZb8=
2021-03-03T14:03:40.082 INFO:teuthology.kill:2021-03-03 14:03:40,080.080 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-03T14:03:40.083 INFO:teuthology.kill:2021-03-03 14:03:40,082.082 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-03T14:03:40.142 INFO:teuthology.kill:2021-03-03 14:03:40,142.142 INFO:teuthology.orchestra.console:Power off smithi028
2021-03-03T14:03:40.156 INFO:teuthology.kill:2021-03-03 14:03:40,155.155 INFO:teuthology.orchestra.console:Power off smithi094
2021-03-03T14:03:48.508 INFO:teuthology.kill:2021-03-03 14:03:48,508.508 INFO:teuthology.orchestra.console:Power off for smithi028 completed
2021-03-03T14:03:48.514 INFO:teuthology.kill:2021-03-03 14:03:48,514.514 INFO:teuthology.orchestra.console:Power off for smithi094 completed
2021-03-03T14:03:48.740 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-03-03T14:03:48.741 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-03-03T14:03:48.741 INFO:teuthology.misc:Compressing logs...
2021-03-03T14:03:48.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-03-03T14:04:48.796 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/misc.py", line 1374, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-03-03T14:04:48.852 INFO:teuthology.kill:No teuthology processes running
2021-03-03T14:04:49.881 INFO:teuthology.kill:Nuking machines: ['smithi028', 'smithi094']
2021-03-03T14:04:50.790 INFO:teuthology.kill:2021-03-03 14:04:50,790.790 INFO:teuthology.nuke:targets:
2021-03-03T14:04:50.791 INFO:teuthology.kill:  smithi028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncUUtrG1RI8lAzaYw0j/mRcmAAEyGH78qKCx8GDgcHHw9eOgnQLmeK4Q5pdIexgDscoc7gwe4lsd1J6XOBDuCF7oDaRClhmEsfyETMHQJAZC5feUBQmieOwbB3LG+sCT2FjaAtVRi3rtW498lRQc+brGoU9v9RQ8+sgzvojf7q090bW170OniLX2W9QxBnUaqPJPFU5ilPURivTdwkY3hxgA2WXbo9amwd3lkJFcAxtNZUItF/ESsFW1X3HJA61pRch7+UjXDkXkxMSCEAmb9yohmJXpXJf9+6kwdtzbwwPlIHx74bifY+4RefothP4Yj/w+l9mNYXi5Ost7ZfmA/gcz7J77tiiPMvhQpkgKW+MNn3EWxGIOEl1bdlYdpATJ0Q110IFk701UA02wMZBhdjeZpPVOIOIdRLUYggBSyzKS+pH88Mevl/xnid7V8GkHv9WYlxOwxJi75+6Q4TygMp0kt8QznFtkm0ONk8ejZ6tehmeSkcQvyhJIkxxR93wU=
2021-03-03T14:04:50.791 INFO:teuthology.kill:  smithi094.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDP4Mpn0iSw63LihMYHEq8WCkbqDaXxGzdfnr/K11VupCbt1T2BarUrvCJJXp/ew7QNm4cqxEK5L1XMv5yIPQMvhIAut69buY83wfJV8QutM0fTlXBYdNlDPjrxKZ4hNh8B2+h9L+WFi+9tKrT5SLt6dhrmfbUW8FRvXS/QJYoY7ylyJhfg72G+BEXd6Kf7LQclrtyiZ3qbKmcs5f5oDi6AQkmCD44dpCSqsylvpuesAaIVxIh+p/yqzReRuTTDwNwdOXjMMEdVbWHv82v8aiPVD7NyZQmWV9ZPRc1qckz8Krv6vpqid2Sig1dMJNah1PWFI7boj1+IaTk9B3o1tf5jG9ubH6CLigtvEI0go8NFmhnqbrv+d6vvgI02qmmF0qYDETx59HMy7i1UWSUXuUG0g1vkk/J+ozRqa9InURfU58iWXVpoXOvgGVxT6FNiRhYZO8IedcsEgPUfye3QA3bq6BfthGVHNv027MqQw/k7yov8B8TCaqcFLtyrx+qiZb8=
2021-03-03T14:04:50.811 INFO:teuthology.kill:2021-03-03 14:04:50,811.811 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-03T14:04:50.814 INFO:teuthology.kill:2021-03-03 14:04:50,814.814 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-03-03T14:04:50.870 INFO:teuthology.kill:2021-03-03 14:04:50,870.870 INFO:teuthology.orchestra.console:Power off smithi094
2021-03-03T14:04:50.882 INFO:teuthology.kill:2021-03-03 14:04:50,882.882 INFO:teuthology.orchestra.console:Power off smithi028
2021-03-03T14:04:51.015 INFO:teuthology.kill:2021-03-03 14:04:51,014.014 INFO:teuthology.orchestra.console:Power off for smithi094 completed
2021-03-03T14:04:51.027 INFO:teuthology.kill:2021-03-03 14:04:51,027.027 INFO:teuthology.orchestra.console:Power off for smithi028 completed
2021-03-03T14:04:51.155 INFO:teuthology.kill:2021-03-03 14:04:51,154.154 INFO:teuthology.lock.ops:unlocked smithi094.front.sepia.ceph.com
2021-03-03T14:04:51.165 INFO:teuthology.kill:2021-03-03 14:04:51,165.165 INFO:teuthology.lock.ops:unlocked smithi028.front.sepia.ceph.com
2021-03-03T14:06:51.425 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15