2021-09-08T00:50:45.595 INFO:root:teuthology version: 1.1.0-b135a4f8
2021-09-08T00:50:45.691 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi161.front.sepia.ceph.com' reimaging
2021-09-08T00:50:45.692 INFO:teuthology.lock.ops:Updating [ubuntu@smithi161.front.sepia.ceph.com]: reset os type and version on server
2021-09-08T00:50:45.692 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2021-09-08T00:50:45.708 INFO:teuthology.lock.ops:Node 'ubuntu@smithi161.front.sepia.ceph.com' reimaging is complete
2021-09-08T00:50:45.709 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi191.front.sepia.ceph.com' reimaging
2021-09-08T00:50:45.709 INFO:teuthology.lock.ops:Updating [ubuntu@smithi191.front.sepia.ceph.com]: reset os type and version on server
2021-09-08T00:50:45.709 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2021-09-08T00:50:45.725 INFO:teuthology.lock.ops:Node 'ubuntu@smithi191.front.sepia.ceph.com' reimaging is complete
2021-09-08T00:50:45.917 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of rhel 8.4
2021-09-08T00:50:45.926 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of rhel 8.4
2021-09-08T00:50:46.314 INFO:teuthology.orchestra.console:Power off smithi161
2021-09-08T00:50:46.330 INFO:teuthology.orchestra.console:Power off smithi191
2021-09-08T00:50:54.693 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2021-09-08T00:50:54.795 INFO:teuthology.orchestra.console:Power on smithi191
2021-09-08T00:50:58.797 INFO:teuthology.orchestra.console:Power off for smithi161 completed
2021-09-08T00:50:58.899 INFO:teuthology.orchestra.console:Power on smithi161
2021-09-08T00:51:03.152 INFO:teuthology.orchestra.console:Power on for smithi161 completed
2021-09-08T00:51:03.164 INFO:teuthology.orchestra.console:Power on for smithi191 completed
2021-09-08T00:51:03.254 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish
2021-09-08T00:51:03.266 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish
2021-09-08T00:55:10.029 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-09-08T00:55:27.131 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-09-08T00:55:30.586 INFO:teuthology.provision.fog.smithi191:Node is ready
2021-09-08T00:55:30.607 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com
2021-09-08T00:55:30.666 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191
2021-09-08T00:55:31.020 INFO:teuthology.provision.fog.smithi191:Deploy complete!
2021-09-08T00:55:46.963 INFO:teuthology.provision.fog.smithi161:Node is ready
2021-09-08T00:55:46.983 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com
2021-09-08T00:55:47.041 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161
2021-09-08T00:55:47.376 INFO:teuthology.provision.fog.smithi161:Deploy complete!
2021-09-08T00:55:47.508 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com
2021-09-08T00:55:47.509 INFO:teuthology.lock.ops:New key found. Updating...
2021-09-08T00:55:47.526 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com
2021-09-08T00:55:47.526 INFO:teuthology.lock.ops:New key found. Updating...
2021-09-08T00:55:47.541 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server
2021-09-08T00:55:47.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-09-08T00:55:47.766 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64
2021-09-08T00:55:47.821 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux"
2021-09-08T00:55:47.821 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.4 (Ootpa)"
2021-09-08T00:55:47.821 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel"
2021-09-08T00:55:47.821 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora"
2021-09-08T00:55:47.822 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.4"
2021-09-08T00:55:47.822 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8"
2021-09-08T00:55:47.822 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2021-09-08T00:55:47.823 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31"
2021-09-08T00:55:47.823 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2021-09-08T00:55:47.823 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/"
2021-09-08T00:55:47.824 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2021-09-08T00:55:47.824 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-09-08T00:55:47.824 INFO:teuthology.orchestra.run.smithi161.stdout:
2021-09-08T00:55:47.824 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-09-08T00:55:47.825 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2021-09-08T00:55:47.825 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-09-08T00:55:47.825 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2021-09-08T00:55:47.826 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2021-09-08T00:55:47.844 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server
2021-09-08T00:55:47.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-09-08T00:55:48.080 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64
2021-09-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux"
2021-09-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.4 (Ootpa)"
2021-09-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel"
2021-09-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora"
2021-09-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.4"
2021-09-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8"
2021-09-08T00:55:48.137 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2021-09-08T00:55:48.137 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31"
2021-09-08T00:55:48.137 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2021-09-08T00:55:48.138 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/"
2021-09-08T00:55:48.138 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2021-09-08T00:55:48.138 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-09-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi191.stdout:
2021-09-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-09-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2021-09-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-09-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2021-09-08T00:55:48.140 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2021-09-08T00:55:48.234 INFO:teuthology.dispatcher.supervisor:Running job 6379117
2021-09-08T00:55:48.235 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b135a4f82b02b966053efba0501e6a47c9d2d793/virtualenv/bin/teuthology -v --owner scheduled_ivancich@teuthology --archive /home/teuthworker/archive/ivancich-2021-09-07_23:07:34-rgw-wip-eric-testing-3-distro-basic-smithi/6379117 --name ivancich-2021-09-07_23:07:34-rgw-wip-eric-testing-3-distro-basic-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability omap_limits overrides realms/three-zone-plus-pubsub supported-random-distro$/{rhel_8} tasks/test_multi} -- /home/teuthworker/archive/ivancich-2021-09-07_23:07:34-rgw-wip-eric-testing-3-distro-basic-smithi/6379117/orig.config.yaml
2021-09-08T00:55:48.238 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ivancich-2021-09-07_23:07:34-rgw-wip-eric-testing-3-distro-basic-smithi/6379117
2021-09-08T00:55:48.239 INFO:teuthology.dispatcher.supervisor:Job PID: 8671
2021-09-08T00:55:48.239 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-09-08T12:56:32.416 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-09-08T12:56:32.613 INFO:teuthology.kill:Killing Pids: {8671}
2021-09-08T12:56:33.684 INFO:teuthology.kill:Nuking machines: ['smithi161', 'smithi191']
2021-09-08T12:56:34.660 INFO:teuthology.kill:2021-09-08 12:56:34,660.660 INFO:teuthology.nuke:targets:
2021-09-08T12:56:34.661 INFO:teuthology.kill:  smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8/+IokkQ8/CAZuh8RZ5BMnUERndoT8OrJZ0pJ47x3q2MWFLvJCRMp8mWuQfn9LslCUbyGH8OcWK/u1qMkUX/GrbOIrB+2OpgQh/ckeOs1Mk+TepQ/JRrxTmWowAWaLkepNmJsp7ZwDGVarLmvrzsqcZNxBYshBTPSOWirLHhunZ6Mdu3Sh2fDQZrIjCT1SdCU3ySiEoaK+ww+WdTwkfw6g1/ZE+hbeKl9QK6CLMf0aHa93IwzKOWrZNOsg+sODFNu0yWzzoz4X3Cs4Pql588J/UMpOq+nLRfqjiNY+2BGDQuQulx9KE8QF3mmgWDUrfVPFfmSLJTVMALXNr+OrInsDIC4kBA33KPhfHeYzWc4DGSrTHfUiNJLqcT2ViOC5Jv3+WO0hVwBkBMsSDKsrHTOslvTizTuhQ9xdMXtoPentqnEhAYNIqvhi4ov9TbulLgJbtyBCPzD/517SCSlyPtC7jP0U2j8lsdqtCy6vmwprxHOOWrrCkEjIpueu0PzFQE=
2021-09-08T12:56:34.661 INFO:teuthology.kill:  smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhRmbtNLlLjcN0Gw+eCDqkOWPanA6waa3rMh++G+6idEdTkaGzByG42vMMnUwflw2VKwpB6EtParo7OKJywMGrg+RDMIP0QPJ+pKXlJKlY4OycmCgj+VV0XHCQVGHIqS6lf/OpwdCQKKcrU796dabVGm34uFzO0YhZTNbiKep+RL+MfIkuK6hG/raUCU4Ey0YsXsERx3yCi7TqxigL0F/bj3cy5UT6Jl2HirrcjCCnuZlld5p8e4GGGFom52MpKtjLl+kgQSfPw+ibq9gqdfHcic/YfxUiZXhWsLViF4T+8SMt+oZaa8oBdwK/Lu4qzsol0UwQdC1FIXIh2zF7r3acVNib3Sm8BGFTei+F14Wm8Zjoi1MK0VinfP93v/Ea9B9b0Hlrx0Y2RWjnk9u/BQdESN3MXB4uZA4tovAIBRNXMdxxEbuLYETUr04XcC4YKG2n20h8Z82kPNdKQ7FTnPQJORrfhmI63zGEYxmjvogYYxJxrUV5hM6EyCVpSnwI4Yk=
2021-09-08T12:56:34.662 INFO:teuthology.kill:2021-09-08 12:56:34,660.660 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-09-08T12:56:34.663 INFO:teuthology.kill:2021-09-08 12:56:34,663.663 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-09-08T12:56:34.725 INFO:teuthology.kill:2021-09-08 12:56:34,725.725 INFO:teuthology.orchestra.console:Power off smithi161
2021-09-08T12:56:34.738 INFO:teuthology.kill:2021-09-08 12:56:34,738.738 INFO:teuthology.orchestra.console:Power off smithi191
2021-09-08T12:56:47.202 INFO:teuthology.kill:2021-09-08 12:56:47,202.202 INFO:teuthology.orchestra.console:Power off for smithi161 completed
2021-09-08T12:56:47.207 INFO:teuthology.kill:2021-09-08 12:56:47,207.207 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2021-09-08T12:56:47.408 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['c1.mon.a', 'c1.mgr.x', 'c1.osd.0', 'c1.osd.1', 'c1.osd.2', 'c1.client.0', 'c1.client.1']
2021-09-08T12:56:47.409 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['c2.mon.a', 'c2.mgr.x', 'c2.osd.0', 'c2.osd.1', 'c2.osd.2', 'c2.client.0', 'c2.client.1']
2021-09-08T12:56:47.409 INFO:teuthology.misc:Compressing logs...
2021-09-08T12:56:47.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-09-08T12:57:47.469 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b135a4f82b02b966053efba0501e6a47c9d2d793/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b135a4f82b02b966053efba0501e6a47c9d2d793/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b135a4f82b02b966053efba0501e6a47c9d2d793/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b135a4f82b02b966053efba0501e6a47c9d2d793/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b135a4f82b02b966053efba0501e6a47c9d2d793/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b135a4f82b02b966053efba0501e6a47c9d2d793/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b135a4f82b02b966053efba0501e6a47c9d2d793/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-09-08T12:57:47.519 INFO:teuthology.kill:No teuthology processes running
2021-09-08T12:57:48.538 INFO:teuthology.kill:Nuking machines: ['smithi161', 'smithi191']
2021-09-08T12:57:49.501 INFO:teuthology.kill:2021-09-08 12:57:49,501.501 INFO:teuthology.nuke:targets:
2021-09-08T12:57:49.502 INFO:teuthology.kill:  smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8/+IokkQ8/CAZuh8RZ5BMnUERndoT8OrJZ0pJ47x3q2MWFLvJCRMp8mWuQfn9LslCUbyGH8OcWK/u1qMkUX/GrbOIrB+2OpgQh/ckeOs1Mk+TepQ/JRrxTmWowAWaLkepNmJsp7ZwDGVarLmvrzsqcZNxBYshBTPSOWirLHhunZ6Mdu3Sh2fDQZrIjCT1SdCU3ySiEoaK+ww+WdTwkfw6g1/ZE+hbeKl9QK6CLMf0aHa93IwzKOWrZNOsg+sODFNu0yWzzoz4X3Cs4Pql588J/UMpOq+nLRfqjiNY+2BGDQuQulx9KE8QF3mmgWDUrfVPFfmSLJTVMALXNr+OrInsDIC4kBA33KPhfHeYzWc4DGSrTHfUiNJLqcT2ViOC5Jv3+WO0hVwBkBMsSDKsrHTOslvTizTuhQ9xdMXtoPentqnEhAYNIqvhi4ov9TbulLgJbtyBCPzD/517SCSlyPtC7jP0U2j8lsdqtCy6vmwprxHOOWrrCkEjIpueu0PzFQE=
2021-09-08T12:57:49.502 INFO:teuthology.kill:  smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhRmbtNLlLjcN0Gw+eCDqkOWPanA6waa3rMh++G+6idEdTkaGzByG42vMMnUwflw2VKwpB6EtParo7OKJywMGrg+RDMIP0QPJ+pKXlJKlY4OycmCgj+VV0XHCQVGHIqS6lf/OpwdCQKKcrU796dabVGm34uFzO0YhZTNbiKep+RL+MfIkuK6hG/raUCU4Ey0YsXsERx3yCi7TqxigL0F/bj3cy5UT6Jl2HirrcjCCnuZlld5p8e4GGGFom52MpKtjLl+kgQSfPw+ibq9gqdfHcic/YfxUiZXhWsLViF4T+8SMt+oZaa8oBdwK/Lu4qzsol0UwQdC1FIXIh2zF7r3acVNib3Sm8BGFTei+F14Wm8Zjoi1MK0VinfP93v/Ea9B9b0Hlrx0Y2RWjnk9u/BQdESN3MXB4uZA4tovAIBRNXMdxxEbuLYETUr04XcC4YKG2n20h8Z82kPNdKQ7FTnPQJORrfhmI63zGEYxmjvogYYxJxrUV5hM6EyCVpSnwI4Yk=
2021-09-08T12:57:49.525 INFO:teuthology.kill:2021-09-08 12:57:49,525.525 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-09-08T12:57:49.528 INFO:teuthology.kill:2021-09-08 12:57:49,528.528 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-09-08T12:57:49.582 INFO:teuthology.kill:2021-09-08 12:57:49,582.582 INFO:teuthology.orchestra.console:Power off smithi161
2021-09-08T12:57:49.626 INFO:teuthology.kill:2021-09-08 12:57:49,626.626 INFO:teuthology.orchestra.console:Power off smithi191
2021-09-08T12:57:49.720 INFO:teuthology.kill:2021-09-08 12:57:49,720.720 INFO:teuthology.orchestra.console:Power off for smithi161 completed
2021-09-08T12:57:49.771 INFO:teuthology.kill:2021-09-08 12:57:49,771.771 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2021-09-08T12:57:49.870 INFO:teuthology.kill:2021-09-08 12:57:49,869.869 INFO:teuthology.lock.ops:unlocked smithi161.front.sepia.ceph.com
2021-09-08T12:57:49.905 INFO:teuthology.kill:2021-09-08 12:57:49,905.905 INFO:teuthology.lock.ops:unlocked smithi191.front.sepia.ceph.com
2021-09-08T12:59:50.228 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15