2021-04-23T12:00:29.480 INFO:root:teuthology version: 1.1.0-2713a3cd
2021-04-23T12:00:29.653 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi003.front.sepia.ceph.com' reimaging
2021-04-23T12:00:29.653 INFO:teuthology.lock.ops:Updating [ubuntu@smithi003.front.sepia.ceph.com]: reset os type and version on server
2021-04-23T12:00:29.654 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2021-04-23T12:00:29.685 INFO:teuthology.lock.ops:Node 'ubuntu@smithi003.front.sepia.ceph.com' reimaging is complete
2021-04-23T12:00:29.685 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi087.front.sepia.ceph.com' reimaging
2021-04-23T12:00:29.686 INFO:teuthology.lock.ops:Updating [ubuntu@smithi087.front.sepia.ceph.com]: reset os type and version on server
2021-04-23T12:00:29.686 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server
2021-04-23T12:00:29.706 INFO:teuthology.lock.ops:Node 'ubuntu@smithi087.front.sepia.ceph.com' reimaging is complete
2021-04-23T12:00:29.890 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of rhel 8.3
2021-04-23T12:00:29.901 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of rhel 8.3
2021-04-23T12:00:30.198 INFO:teuthology.orchestra.console:Power off smithi003
2021-04-23T12:00:30.223 INFO:teuthology.orchestra.console:Power off smithi087
2021-04-23T12:00:30.352 INFO:teuthology.orchestra.console:Power off for smithi003 completed
2021-04-23T12:00:30.359 INFO:teuthology.orchestra.console:Power off for smithi087 completed
2021-04-23T12:00:30.453 INFO:teuthology.orchestra.console:Power on smithi003
2021-04-23T12:00:30.460 INFO:teuthology.orchestra.console:Power on smithi087
2021-04-23T12:00:34.704 INFO:teuthology.orchestra.console:Power on for smithi003 completed
2021-04-23T12:00:34.804 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish
2021-04-23T12:00:38.819 INFO:teuthology.orchestra.console:Power on for smithi087 completed
2021-04-23T12:00:38.920 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish
2021-04-23T12:04:45.131 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-23T12:04:51.372 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-04-23T12:04:51.708 INFO:teuthology.provision.fog.smithi003:Node is ready
2021-04-23T12:04:51.723 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com
2021-04-23T12:04:51.778 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003
2021-04-23T12:04:52.204 INFO:teuthology.provision.fog.smithi003:Deploy complete!
2021-04-23T12:05:05.852 INFO:teuthology.provision.fog.smithi087:Node is ready
2021-04-23T12:05:05.872 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com
2021-04-23T12:05:05.929 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087
2021-04-23T12:05:06.291 INFO:teuthology.provision.fog.smithi087:Deploy complete!
2021-04-23T12:05:06.405 INFO:teuthology.lock.ops:Checking smithi003.front.sepia.ceph.com
2021-04-23T12:05:06.406 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-23T12:05:06.422 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com
2021-04-23T12:05:06.422 INFO:teuthology.lock.ops:New key found. Updating...
2021-04-23T12:05:06.437 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: set os type and version on server
2021-04-23T12:05:06.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-23T12:05:06.733 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="Red Hat Enterprise Linux"
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="8.3 (Ootpa)"
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:ID="rhel"
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="fedora"
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="8.3"
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el8"
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31"
2021-04-23T12:05:06.788 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-04-23T12:05:06.789 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://www.redhat.com/"
2021-04-23T12:05:06.789 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-04-23T12:05:06.789 INFO:teuthology.orchestra.run.smithi003.stdout:
2021-04-23T12:05:06.789 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-04-23T12:05:06.789 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-04-23T12:05:06.789 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-04-23T12:05:06.789 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-04-23T12:05:06.790 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2021-04-23T12:05:06.805 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server
2021-04-23T12:05:06.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-23T12:05:07.092 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64
2021-04-23T12:05:07.146 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux"
2021-04-23T12:05:07.146 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.3 (Ootpa)"
2021-04-23T12:05:07.146 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel"
2021-04-23T12:05:07.146 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora"
2021-04-23T12:05:07.146 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.3"
2021-04-23T12:05:07.146 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8"
2021-04-23T12:05:07.146 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31"
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/"
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-04-23T12:05:07.147 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-04-23T12:05:07.148 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server
2021-04-23T12:05:07.221 INFO:teuthology.dispatcher.supervisor:Running job 6064400
2021-04-23T12:05:07.221 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-04-22_11:34:56-rados-no_column_b-distro-basic-smithi/6064400 --name benhanokh-2021-04-22_11:34:56-rados-no_column_b-distro-basic-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/small-objects-localized} -- /home/teuthworker/archive/benhanokh-2021-04-22_11:34:56-rados-no_column_b-distro-basic-smithi/6064400/orig.config.yaml
2021-04-23T12:05:07.225 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-04-22_11:34:56-rados-no_column_b-distro-basic-smithi/6064400
2021-04-23T12:05:07.225 INFO:teuthology.dispatcher.supervisor:Job PID: 14589
2021-04-23T12:05:07.225 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-04-24T00:06:27.982 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-04-24T00:06:28.154 INFO:teuthology.kill:Killing Pids: {14589}
2021-04-24T00:06:29.557 INFO:teuthology.kill:Nuking machines: ['smithi003', 'smithi087']
2021-04-24T00:06:30.493 INFO:teuthology.kill:2021-04-24 00:06:30,493.493 INFO:teuthology.nuke:targets:
2021-04-24T00:06:30.494 INFO:teuthology.kill:  smithi003.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+f1MSDN4j7/euCqgGuT2MQEWlBlNkeZJkDWCsrIi+ka7cMOFHP9mtzmPX8s9tNZ3D2eRXCZ2HV8BL1aDKyn8YbAC0aGy/KS/ZjY0zyq6PoI85agld9Rg/GWs2XdU829PyZgtG7C5LQ/ilMzIGEj+oMQwEkOjXgM+ZZ4BxWj8UXnO8cUkKdBoq8rS5RzlV3BcY70u3MH/kjhWgiinIOtz/tJhJ8xBQf5sim4Vr5Vv8bZZVqcfDWLc7vfeJZqNDdXwCQ1FRs0PbJd5ZsqJb2b5QIiZIttKAd8jd7mJtk/GSVdDjsEkouITazUHGHLtE3Ax9myOveKXQ5rN8/W2Rl3oHt2TT9rYkalPCbrXQ/+zdUKu3CFHscfXvgEDiMGaHTv+Fc4uWlbToXhWnI0yP2cP52OrbiBeLnKq2fcmn14AFJ1rcVB03wE0Y7kEOZXjOd7+xZcBNxLm3x67eKc06UsInYNmoWMM2NsTx7J2y2cjXbWjd13BAcN5DrxOGC8NOgcc=
2021-04-24T00:06:30.494 INFO:teuthology.kill:  smithi087.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSyZqKzwge0ofUdoYyJSwdHofedj9h95dw2YhUNcpiZKpex2EUm3YVvk/l7d3lnv1GFOkT9PCN5nJTAd8vz9jyFGgcbFyKNHSP6gJDMbg2gopf5H/v0Ddi9wIudaQJhdH+4by6zBEr+uPwOFNrkvEnpiZXb616Gk1EBkh3ToAQ4ME6AxaEVSfK0R4ToWnepZ1DiZ4rzaePVzwIw7hnBbVCoBdu56K+VGKXen6KINp14TZM6nr6y9QoUrHtI/XlAwu1ZDlsW3ilLmgNLhcfyYjHWrzhheYgPoyFKGKC+m6WrrYfuakxV2YYujFEgXPIZeUhxDmMhbJu9y9nvbvSTsuyinTv3kRe27fldeoUKZ3tyMiNSqQwnJwvPgevwfbnPCVmAYR7AfXIAExtAKiSfYpxCN4+3mN2PnhPpzbGEd0RsazcWEkkje6+W8BFzne1qqgwTVU2YNrQlWG39thCGS7CbaOLlk0k6DC2ueVop3wqeECHgrQwpk4bdjp7/KiRdl8=
2021-04-24T00:06:30.494 INFO:teuthology.kill:2021-04-24 00:06:30,493.493 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-24T00:06:30.496 INFO:teuthology.kill:2021-04-24 00:06:30,496.496 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-24T00:06:30.715 INFO:teuthology.kill:2021-04-24 00:06:30,715.715 INFO:teuthology.orchestra.console:Power off smithi003
2021-04-24T00:06:30.728 INFO:teuthology.kill:2021-04-24 00:06:30,728.728 INFO:teuthology.orchestra.console:Power off smithi087
2021-04-24T00:06:39.082 INFO:teuthology.kill:2021-04-24 00:06:39,082.082 INFO:teuthology.orchestra.console:Power off for smithi003 completed
2021-04-24T00:06:43.203 INFO:teuthology.kill:2021-04-24 00:06:43,203.203 INFO:teuthology.orchestra.console:Power off for smithi087 completed
2021-04-24T00:06:43.452 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-04-24T00:06:43.452 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-04-24T00:06:43.452 INFO:teuthology.misc:Compressing logs...
2021-04-24T00:06:43.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-04-24T00:07:18.107 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/remote.py", line 215, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_2713a3cd31b17738a50039eaa9d859b5dc39fb8a/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-04-24T00:07:18.168 INFO:teuthology.kill:No teuthology processes running
2021-04-24T00:07:19.269 INFO:teuthology.kill:Nuking machines: ['smithi003', 'smithi087']
2021-04-24T00:07:20.178 INFO:teuthology.kill:2021-04-24 00:07:20,177.177 INFO:teuthology.nuke:targets:
2021-04-24T00:07:20.178 INFO:teuthology.kill:  smithi003.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+f1MSDN4j7/euCqgGuT2MQEWlBlNkeZJkDWCsrIi+ka7cMOFHP9mtzmPX8s9tNZ3D2eRXCZ2HV8BL1aDKyn8YbAC0aGy/KS/ZjY0zyq6PoI85agld9Rg/GWs2XdU829PyZgtG7C5LQ/ilMzIGEj+oMQwEkOjXgM+ZZ4BxWj8UXnO8cUkKdBoq8rS5RzlV3BcY70u3MH/kjhWgiinIOtz/tJhJ8xBQf5sim4Vr5Vv8bZZVqcfDWLc7vfeJZqNDdXwCQ1FRs0PbJd5ZsqJb2b5QIiZIttKAd8jd7mJtk/GSVdDjsEkouITazUHGHLtE3Ax9myOveKXQ5rN8/W2Rl3oHt2TT9rYkalPCbrXQ/+zdUKu3CFHscfXvgEDiMGaHTv+Fc4uWlbToXhWnI0yP2cP52OrbiBeLnKq2fcmn14AFJ1rcVB03wE0Y7kEOZXjOd7+xZcBNxLm3x67eKc06UsInYNmoWMM2NsTx7J2y2cjXbWjd13BAcN5DrxOGC8NOgcc=
2021-04-24T00:07:20.178 INFO:teuthology.kill:  smithi087.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSyZqKzwge0ofUdoYyJSwdHofedj9h95dw2YhUNcpiZKpex2EUm3YVvk/l7d3lnv1GFOkT9PCN5nJTAd8vz9jyFGgcbFyKNHSP6gJDMbg2gopf5H/v0Ddi9wIudaQJhdH+4by6zBEr+uPwOFNrkvEnpiZXb616Gk1EBkh3ToAQ4ME6AxaEVSfK0R4ToWnepZ1DiZ4rzaePVzwIw7hnBbVCoBdu56K+VGKXen6KINp14TZM6nr6y9QoUrHtI/XlAwu1ZDlsW3ilLmgNLhcfyYjHWrzhheYgPoyFKGKC+m6WrrYfuakxV2YYujFEgXPIZeUhxDmMhbJu9y9nvbvSTsuyinTv3kRe27fldeoUKZ3tyMiNSqQwnJwvPgevwfbnPCVmAYR7AfXIAExtAKiSfYpxCN4+3mN2PnhPpzbGEd0RsazcWEkkje6+W8BFzne1qqgwTVU2YNrQlWG39thCGS7CbaOLlk0k6DC2ueVop3wqeECHgrQwpk4bdjp7/KiRdl8=
2021-04-24T00:07:20.198 INFO:teuthology.kill:2021-04-24 00:07:20,197.197 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-24T00:07:20.209 INFO:teuthology.kill:2021-04-24 00:07:20,209.209 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-04-24T00:07:20.325 INFO:teuthology.kill:2021-04-24 00:07:20,324.324 INFO:teuthology.orchestra.console:Power off smithi003
2021-04-24T00:07:20.369 INFO:teuthology.kill:2021-04-24 00:07:20,368.368 INFO:teuthology.orchestra.console:Power off smithi087
2021-04-24T00:07:20.477 INFO:teuthology.kill:2021-04-24 00:07:20,477.477 INFO:teuthology.orchestra.console:Power off for smithi003 completed
2021-04-24T00:07:20.508 INFO:teuthology.kill:2021-04-24 00:07:20,508.508 INFO:teuthology.orchestra.console:Power off for smithi087 completed
2021-04-24T00:07:20.619 INFO:teuthology.kill:2021-04-24 00:07:20,618.618 INFO:teuthology.lock.ops:unlocked smithi003.front.sepia.ceph.com
2021-04-24T00:07:20.649 INFO:teuthology.kill:2021-04-24 00:07:20,649.649 INFO:teuthology.lock.ops:unlocked smithi087.front.sepia.ceph.com
2021-04-24T00:09:20.928 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-04-24T00:09:21.018 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-04-24T00:09:21.019 INFO:teuthology.nuke:Checking targets against current locks
2021-04-24T00:09:21.094 INFO:teuthology.nuke:Not nuking smithi087.front.sepia.ceph.com because description doesn't match
2021-04-24T00:09:21.095 INFO:teuthology.nuke:Not nuking smithi003.front.sepia.ceph.com because description doesn't match