2021-07-27T22:21:04.185 INFO:root:teuthology version: 1.1.0-73aa7e3b
2021-07-27T22:21:04.301 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi191.front.sepia.ceph.com' reimaging
2021-07-27T22:21:04.302 INFO:teuthology.lock.ops:Updating [ubuntu@smithi191.front.sepia.ceph.com]: reset os type and version on server
2021-07-27T22:21:04.302 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2021-07-27T22:21:04.318 INFO:teuthology.lock.ops:Node 'ubuntu@smithi191.front.sepia.ceph.com' reimaging is complete
2021-07-27T22:21:04.318 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi194.front.sepia.ceph.com' reimaging
2021-07-27T22:21:04.318 INFO:teuthology.lock.ops:Updating [ubuntu@smithi194.front.sepia.ceph.com]: reset os type and version on server
2021-07-27T22:21:04.319 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server
2021-07-27T22:21:04.335 INFO:teuthology.lock.ops:Node 'ubuntu@smithi194.front.sepia.ceph.com' reimaging is complete
2021-07-27T22:21:04.522 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of rhel 8.4
2021-07-27T22:21:04.536 INFO:teuthology.provision.fog.smithi194:Scheduling deploy of rhel 8.4
2021-07-27T22:21:04.856 INFO:teuthology.orchestra.console:Power off smithi194
2021-07-27T22:21:04.874 INFO:teuthology.orchestra.console:Power off smithi191
2021-07-27T22:21:13.237 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2021-07-27T22:21:13.338 INFO:teuthology.orchestra.console:Power on smithi191
2021-07-27T22:21:17.344 INFO:teuthology.orchestra.console:Power off for smithi194 completed
2021-07-27T22:21:17.446 INFO:teuthology.orchestra.console:Power on smithi194
2021-07-27T22:21:21.701 INFO:teuthology.orchestra.console:Power on for smithi194 completed
2021-07-27T22:21:21.740 INFO:teuthology.orchestra.console:Power on for smithi191 completed
2021-07-27T22:21:21.803 INFO:teuthology.provision.fog.smithi194:Waiting for deploy to finish
2021-07-27T22:21:21.841 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish
2021-07-27T22:25:53.670 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-27T22:25:53.887 INFO:teuthology.provision.fog.smithi191:Node is ready
2021-07-27T22:25:53.903 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com
2021-07-27T22:25:53.965 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191
2021-07-27T22:25:54.274 INFO:teuthology.provision.fog.smithi191:Deploy complete!
2021-07-27T22:26:08.760 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-07-27T22:26:08.998 INFO:teuthology.provision.fog.smithi194:Node is ready
2021-07-27T22:26:09.014 INFO:teuthology.orchestra.run.smithi194.stdout:smithi194.front.sepia.ceph.com
2021-07-27T22:26:09.070 INFO:teuthology.orchestra.run.smithi194.stdout:172.21.15.194 smithi194.front.sepia.ceph.com smithi194
2021-07-27T22:26:09.390 INFO:teuthology.provision.fog.smithi194:Deploy complete!
2021-07-27T22:26:09.540 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com
2021-07-27T22:26:09.541 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-27T22:26:09.558 INFO:teuthology.lock.ops:Checking smithi194.front.sepia.ceph.com
2021-07-27T22:26:09.558 INFO:teuthology.lock.ops:New key found. Updating...
2021-07-27T22:26:09.577 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server
2021-07-27T22:26:09.577 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-27T22:26:09.813 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64
2021-07-27T22:26:09.868 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux"
2021-07-27T22:26:09.869 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.4 (Ootpa)"
2021-07-27T22:26:09.869 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel"
2021-07-27T22:26:09.869 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora"
2021-07-27T22:26:09.869 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.4"
2021-07-27T22:26:09.870 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8"
2021-07-27T22:26:09.870 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2021-07-27T22:26:09.870 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31"
2021-07-27T22:26:09.870 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2021-07-27T22:26:09.871 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/"
2021-07-27T22:26:09.871 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2021-07-27T22:26:09.871 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-27T22:26:09.871 INFO:teuthology.orchestra.run.smithi191.stdout:
2021-07-27T22:26:09.873 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-27T22:26:09.873 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2021-07-27T22:26:09.873 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-07-27T22:26:09.874 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2021-07-27T22:26:09.875 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2021-07-27T22:26:09.890 INFO:teuthology.lock.ops:Updating [smithi194.front.sepia.ceph.com]: set os type and version on server
2021-07-27T22:26:09.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-27T22:26:10.114 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64
2021-07-27T22:26:10.168 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="Red Hat Enterprise Linux"
2021-07-27T22:26:10.168 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="8.4 (Ootpa)"
2021-07-27T22:26:10.169 INFO:teuthology.orchestra.run.smithi194.stdout:ID="rhel"
2021-07-27T22:26:10.169 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="fedora"
2021-07-27T22:26:10.169 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="8.4"
2021-07-27T22:26:10.169 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el8"
2021-07-27T22:26:10.170 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2021-07-27T22:26:10.170 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31"
2021-07-27T22:26:10.170 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2021-07-27T22:26:10.170 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://www.redhat.com/"
2021-07-27T22:26:10.170 INFO:teuthology.orchestra.run.smithi194.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2021-07-27T22:26:10.171 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-07-27T22:26:10.171 INFO:teuthology.orchestra.run.smithi194.stdout:
2021-07-27T22:26:10.171 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-07-27T22:26:10.171 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2021-07-27T22:26:10.172 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-07-27T22:26:10.172 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2021-07-27T22:26:10.172 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server
2021-07-27T22:26:10.257 INFO:teuthology.dispatcher.supervisor:Running job 6296437
2021-07-27T22:26:10.258 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296437 --name benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296437/orig.config.yaml
2021-07-27T22:26:10.261 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-07-27_13:23:42-rados-wip_gbenhano_ncb_fix2-distro-basic-smithi/6296437
2021-07-27T22:26:10.262 INFO:teuthology.dispatcher.supervisor:Job PID: 12166
2021-07-27T22:26:10.262 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-07-28T10:27:00.339 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-07-28T10:27:00.560 INFO:teuthology.kill:Killing Pids: {12166}
2021-07-28T10:27:01.785 INFO:teuthology.kill:Nuking machines: ['smithi191', 'smithi194']
2021-07-28T10:27:02.820 INFO:teuthology.kill:2021-07-28 10:27:02,819.819 INFO:teuthology.nuke:targets:
2021-07-28T10:27:02.821 INFO:teuthology.kill:  smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYL49nb7D/sL7efG86HnoQtMRtzYd8YTDqs0JUEUlahosMKe2jE5kwR3X5oEbng/+xIQAmnlFr/LxNZYFHZr4rslVY7Af5Akc2FUa5/r4U7gbxD+OOWi4wYcCJkLHJFFB29PX1ZnMaBE2wrDg431T/zdsBiXS9NHryOqOLIAGhVPS4eBVdVmtaEZEAv9IKgBo+e1J+sOiRe+zonyKXFLEIP5EqHRxmIm/2Okye29l1iiZQ3Z4H3zduxZcSVsuv0A82sLwSKHQAreKOuNgXC114iyMAXnnT/KUNDRgYnczJuUwZW1yBSlMPcvNPng751WwHwmKztTTAKjLGcoWIboU++nN2IxF11QJ5WfJ7jq8ggE1TWRzOowKh99KoYt0xbJRAZGSpKlJ1ECaqqjSnFUNaSORhvXYe5FUY9TJUBot1h6o2DQ+M4WcyVMas/k3JnEwNi/Dlo9cTty4xZh6p6wVjRhRgtYVnZk3/Kf+37CRWFE7C4TiVLQeStrkEOgQCvtc=
2021-07-28T10:27:02.821 INFO:teuthology.kill:  smithi194.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5FTUXKgn1ppgFbVuBRJlX6wOvKLHZtJdTt4rKPTkjRMKW0imqzFSb1OaMxmg05J3pGjFlYlfDUmDKi48ujcOdRgtLgfQjwvde83TsuISCCeieNRQAq27bkkAtZUNGp253IzvEWGcOgE0Vc3lnq8MNLFBO1TYO7BJL8keDtF5jhWoJ2q6lDWAT+4ozOAqbH8cLeIe43HnE87MMp/uqsZnna9Wx5Lid88jYt4OXney0JO6aeQg0ybqE8oGXuBhdBvhpl5POp89BE6cCE1ZNvIwYfj2uuShEJhcFn0MJoQ341RaU2yBdMJlikJ3jPpjmhSaYV52Czglr+6rgaUIG7/TazE0Kr4IHfHeZlbMJZ6NN8QyTfzd2rLl+GSkxT0xFcS+pcL6cFIw88GFlIziRYXD5NEGyfqPv9/pi6UK6QujPc2zUReKv8JizhxrtOObAMygygkGfr0DyI0SDgwYZe0WBsJQAiWhj2g5+Tz5HgFHZZrUXgRE5DrMiCvue4EzCbxc=
2021-07-28T10:27:02.822 INFO:teuthology.kill:2021-07-28 10:27:02,820.820 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:27:02.823 INFO:teuthology.kill:2021-07-28 10:27:02,822.822 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:27:02.888 INFO:teuthology.kill:2021-07-28 10:27:02,888.888 INFO:teuthology.orchestra.console:Power off smithi191
2021-07-28T10:27:02.905 INFO:teuthology.kill:2021-07-28 10:27:02,905.905 INFO:teuthology.orchestra.console:Power off smithi194
2021-07-28T10:27:11.261 INFO:teuthology.kill:2021-07-28 10:27:11,260.260 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2021-07-28T10:27:15.389 INFO:teuthology.kill:2021-07-28 10:27:15,389.389 INFO:teuthology.orchestra.console:Power off for smithi194 completed
2021-07-28T10:27:15.660 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-07-28T10:27:15.661 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-07-28T10:27:15.662 INFO:teuthology.misc:Compressing logs...
2021-07-28T10:27:15.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-07-28T10:27:50.347 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_73aa7e3b960c7ffac669297b6aa86606265edd1b/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-07-28T10:27:50.409 INFO:teuthology.kill:No teuthology processes running
2021-07-28T10:27:51.591 INFO:teuthology.kill:Nuking machines: ['smithi191', 'smithi194']
2021-07-28T10:27:52.572 INFO:teuthology.kill:2021-07-28 10:27:52,571.571 INFO:teuthology.nuke:targets:
2021-07-28T10:27:52.573 INFO:teuthology.kill:  smithi191.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYL49nb7D/sL7efG86HnoQtMRtzYd8YTDqs0JUEUlahosMKe2jE5kwR3X5oEbng/+xIQAmnlFr/LxNZYFHZr4rslVY7Af5Akc2FUa5/r4U7gbxD+OOWi4wYcCJkLHJFFB29PX1ZnMaBE2wrDg431T/zdsBiXS9NHryOqOLIAGhVPS4eBVdVmtaEZEAv9IKgBo+e1J+sOiRe+zonyKXFLEIP5EqHRxmIm/2Okye29l1iiZQ3Z4H3zduxZcSVsuv0A82sLwSKHQAreKOuNgXC114iyMAXnnT/KUNDRgYnczJuUwZW1yBSlMPcvNPng751WwHwmKztTTAKjLGcoWIboU++nN2IxF11QJ5WfJ7jq8ggE1TWRzOowKh99KoYt0xbJRAZGSpKlJ1ECaqqjSnFUNaSORhvXYe5FUY9TJUBot1h6o2DQ+M4WcyVMas/k3JnEwNi/Dlo9cTty4xZh6p6wVjRhRgtYVnZk3/Kf+37CRWFE7C4TiVLQeStrkEOgQCvtc=
2021-07-28T10:27:52.573 INFO:teuthology.kill:  smithi194.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5FTUXKgn1ppgFbVuBRJlX6wOvKLHZtJdTt4rKPTkjRMKW0imqzFSb1OaMxmg05J3pGjFlYlfDUmDKi48ujcOdRgtLgfQjwvde83TsuISCCeieNRQAq27bkkAtZUNGp253IzvEWGcOgE0Vc3lnq8MNLFBO1TYO7BJL8keDtF5jhWoJ2q6lDWAT+4ozOAqbH8cLeIe43HnE87MMp/uqsZnna9Wx5Lid88jYt4OXney0JO6aeQg0ybqE8oGXuBhdBvhpl5POp89BE6cCE1ZNvIwYfj2uuShEJhcFn0MJoQ341RaU2yBdMJlikJ3jPpjmhSaYV52Czglr+6rgaUIG7/TazE0Kr4IHfHeZlbMJZ6NN8QyTfzd2rLl+GSkxT0xFcS+pcL6cFIw88GFlIziRYXD5NEGyfqPv9/pi6UK6QujPc2zUReKv8JizhxrtOObAMygygkGfr0DyI0SDgwYZe0WBsJQAiWhj2g5+Tz5HgFHZZrUXgRE5DrMiCvue4EzCbxc=
2021-07-28T10:27:52.592 INFO:teuthology.kill:2021-07-28 10:27:52,592.592 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:27:52.595 INFO:teuthology.kill:2021-07-28 10:27:52,595.595 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-07-28T10:27:52.654 INFO:teuthology.kill:2021-07-28 10:27:52,654.654 INFO:teuthology.orchestra.console:Power off smithi191
2021-07-28T10:27:52.671 INFO:teuthology.kill:2021-07-28 10:27:52,670.670 INFO:teuthology.orchestra.console:Power off smithi194
2021-07-28T10:27:52.805 INFO:teuthology.kill:2021-07-28 10:27:52,803.803 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2021-07-28T10:27:52.816 INFO:teuthology.kill:2021-07-28 10:27:52,816.816 INFO:teuthology.orchestra.console:Power off for smithi194 completed
2021-07-28T10:27:52.947 INFO:teuthology.kill:2021-07-28 10:27:52,947.947 INFO:teuthology.lock.ops:unlocked smithi191.front.sepia.ceph.com
2021-07-28T10:27:52.954 INFO:teuthology.kill:2021-07-28 10:27:52,954.954 INFO:teuthology.lock.ops:unlocked smithi194.front.sepia.ceph.com
2021-07-28T10:29:53.250 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-07-28T10:29:53.340 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-07-28T10:29:53.340 INFO:teuthology.nuke:Checking targets against current locks
2021-07-28T10:29:53.416 INFO:teuthology.nuke:Not nuking smithi191.front.sepia.ceph.com because description doesn't match
2021-07-28T10:29:53.417 INFO:teuthology.nuke:Not nuking smithi194.front.sepia.ceph.com because description doesn't match