2021-05-05T19:32:14.288 INFO:root:teuthology version: 1.1.0-19220a3b
2021-05-05T19:32:14.394 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba014.front.sepia.ceph.com' reimaging
2021-05-05T19:32:14.395 INFO:teuthology.lock.ops:Updating [ubuntu@gibba014.front.sepia.ceph.com]: reset os type and version on server
2021-05-05T19:32:14.396 INFO:teuthology.lock.ops:Updating gibba014.front.sepia.ceph.com on lock server
2021-05-05T19:32:14.412 INFO:teuthology.lock.ops:Node 'ubuntu@gibba014.front.sepia.ceph.com' reimaging is complete
2021-05-05T19:32:14.413 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba017.front.sepia.ceph.com' reimaging
2021-05-05T19:32:14.413 INFO:teuthology.lock.ops:Updating [ubuntu@gibba017.front.sepia.ceph.com]: reset os type and version on server
2021-05-05T19:32:14.413 INFO:teuthology.lock.ops:Updating gibba017.front.sepia.ceph.com on lock server
2021-05-05T19:32:14.430 INFO:teuthology.lock.ops:Node 'ubuntu@gibba017.front.sepia.ceph.com' reimaging is complete
2021-05-05T19:32:14.641 INFO:teuthology.provision.fog.gibba014:Scheduling deploy of rhel 8.3
2021-05-05T19:32:14.664 INFO:teuthology.provision.fog.gibba017:Scheduling deploy of rhel 8.3
2021-05-05T19:32:15.037 INFO:teuthology.orchestra.console:Power off gibba014
2021-05-05T19:32:15.081 INFO:teuthology.orchestra.console:Power off gibba017
2021-05-05T19:32:23.425 INFO:teuthology.orchestra.console:Power off for gibba014 completed
2021-05-05T19:32:23.452 INFO:teuthology.orchestra.console:Power off for gibba017 completed
2021-05-05T19:32:23.527 INFO:teuthology.orchestra.console:Power on gibba014
2021-05-05T19:32:23.554 INFO:teuthology.orchestra.console:Power on gibba017
2021-05-05T19:32:31.918 INFO:teuthology.orchestra.console:Power on for gibba014 completed
2021-05-05T19:32:32.003 INFO:teuthology.orchestra.console:Power on for gibba017 completed
2021-05-05T19:32:32.021 INFO:teuthology.provision.fog.gibba014:Waiting for deploy to finish
2021-05-05T19:32:32.104 INFO:teuthology.provision.fog.gibba017:Waiting for deploy to finish
2021-05-05T19:36:58.111 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-05T19:36:58.199 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-05-05T19:36:58.349 INFO:teuthology.provision.fog.gibba017:Node is ready
2021-05-05T19:36:58.360 INFO:teuthology.orchestra.run.gibba017.stdout:gibba017.front.sepia.ceph.com
2021-05-05T19:36:58.412 INFO:teuthology.orchestra.run.gibba017.stdout:172.21.2.117 gibba017.front.sepia.ceph.com gibba017
2021-05-05T19:36:58.424 INFO:teuthology.provision.fog.gibba014:Node is ready
2021-05-05T19:36:58.435 INFO:teuthology.orchestra.run.gibba014.stdout:gibba014.front.sepia.ceph.com
2021-05-05T19:36:58.488 INFO:teuthology.orchestra.run.gibba014.stdout:172.21.2.114 gibba014.front.sepia.ceph.com gibba014
2021-05-05T19:36:58.695 INFO:teuthology.provision.fog.gibba017:Deploy complete!
2021-05-05T19:36:58.779 INFO:teuthology.provision.fog.gibba014:Deploy complete!
2021-05-05T19:36:58.920 INFO:teuthology.lock.ops:Checking gibba014.front.sepia.ceph.com
2021-05-05T19:36:58.921 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-05T19:36:58.939 INFO:teuthology.lock.ops:Checking gibba017.front.sepia.ceph.com
2021-05-05T19:36:58.940 INFO:teuthology.lock.ops:New key found. Updating...
2021-05-05T19:36:58.958 INFO:teuthology.lock.ops:Updating [gibba014.front.sepia.ceph.com]: set os type and version on server
2021-05-05T19:36:58.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-05T19:36:59.247 INFO:teuthology.orchestra.run.gibba014.stdout:x86_64
2021-05-05T19:36:59.299 INFO:teuthology.orchestra.run.gibba014.stdout:NAME="Red Hat Enterprise Linux"
2021-05-05T19:36:59.300 INFO:teuthology.orchestra.run.gibba014.stdout:VERSION="8.3 (Ootpa)"
2021-05-05T19:36:59.300 INFO:teuthology.orchestra.run.gibba014.stdout:ID="rhel"
2021-05-05T19:36:59.300 INFO:teuthology.orchestra.run.gibba014.stdout:ID_LIKE="fedora"
2021-05-05T19:36:59.300 INFO:teuthology.orchestra.run.gibba014.stdout:VERSION_ID="8.3"
2021-05-05T19:36:59.301 INFO:teuthology.orchestra.run.gibba014.stdout:PLATFORM_ID="platform:el8"
2021-05-05T19:36:59.301 INFO:teuthology.orchestra.run.gibba014.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-05-05T19:36:59.301 INFO:teuthology.orchestra.run.gibba014.stdout:ANSI_COLOR="0;31"
2021-05-05T19:36:59.301 INFO:teuthology.orchestra.run.gibba014.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-05-05T19:36:59.302 INFO:teuthology.orchestra.run.gibba014.stdout:HOME_URL="https://www.redhat.com/"
2021-05-05T19:36:59.302 INFO:teuthology.orchestra.run.gibba014.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-05-05T19:36:59.303 INFO:teuthology.orchestra.run.gibba014.stdout:
2021-05-05T19:36:59.303 INFO:teuthology.orchestra.run.gibba014.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-05-05T19:36:59.303 INFO:teuthology.orchestra.run.gibba014.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-05-05T19:36:59.304 INFO:teuthology.orchestra.run.gibba014.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-05-05T19:36:59.304 INFO:teuthology.orchestra.run.gibba014.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-05-05T19:36:59.307 INFO:teuthology.lock.ops:Updating gibba014.front.sepia.ceph.com on lock server
2021-05-05T19:36:59.332 INFO:teuthology.lock.ops:Updating [gibba017.front.sepia.ceph.com]: set os type and version on server
2021-05-05T19:36:59.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-05T19:36:59.583 INFO:teuthology.orchestra.run.gibba017.stdout:x86_64
2021-05-05T19:36:59.637 INFO:teuthology.orchestra.run.gibba017.stdout:NAME="Red Hat Enterprise Linux"
2021-05-05T19:36:59.637 INFO:teuthology.orchestra.run.gibba017.stdout:VERSION="8.3 (Ootpa)"
2021-05-05T19:36:59.638 INFO:teuthology.orchestra.run.gibba017.stdout:ID="rhel"
2021-05-05T19:36:59.638 INFO:teuthology.orchestra.run.gibba017.stdout:ID_LIKE="fedora"
2021-05-05T19:36:59.638 INFO:teuthology.orchestra.run.gibba017.stdout:VERSION_ID="8.3"
2021-05-05T19:36:59.639 INFO:teuthology.orchestra.run.gibba017.stdout:PLATFORM_ID="platform:el8"
2021-05-05T19:36:59.639 INFO:teuthology.orchestra.run.gibba017.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)"
2021-05-05T19:36:59.639 INFO:teuthology.orchestra.run.gibba017.stdout:ANSI_COLOR="0;31"
2021-05-05T19:36:59.640 INFO:teuthology.orchestra.run.gibba017.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA"
2021-05-05T19:36:59.640 INFO:teuthology.orchestra.run.gibba017.stdout:HOME_URL="https://www.redhat.com/"
2021-05-05T19:36:59.640 INFO:teuthology.orchestra.run.gibba017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2021-05-05T19:36:59.641 INFO:teuthology.orchestra.run.gibba017.stdout:
2021-05-05T19:36:59.641 INFO:teuthology.orchestra.run.gibba017.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2021-05-05T19:36:59.641 INFO:teuthology.orchestra.run.gibba017.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3
2021-05-05T19:36:59.641 INFO:teuthology.orchestra.run.gibba017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2021-05-05T19:36:59.642 INFO:teuthology.orchestra.run.gibba017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3"
2021-05-05T19:36:59.642 INFO:teuthology.lock.ops:Updating gibba017.front.sepia.ceph.com on lock server
2021-05-05T19:36:59.823 INFO:teuthology.dispatcher.supervisor:Running job 6099548
2021-05-05T19:36:59.823 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/virtualenv/bin/teuthology -v --owner scheduled_benhanokh@teuthology --archive /home/teuthworker/archive/benhanokh-2021-05-05_16:36:42-rados-benhanokh-NCB-distro-basic-gibba/6099548 --name benhanokh-2021-05-05_16:36:42-rados-benhanokh-NCB-distro-basic-gibba --description rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} -- /home/teuthworker/archive/benhanokh-2021-05-05_16:36:42-rados-benhanokh-NCB-distro-basic-gibba/6099548/orig.config.yaml
2021-05-05T19:36:59.829 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/benhanokh-2021-05-05_16:36:42-rados-benhanokh-NCB-distro-basic-gibba/6099548
2021-05-05T19:36:59.831 INFO:teuthology.dispatcher.supervisor:Job PID: 10209
2021-05-05T19:36:59.831 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-05-06T07:37:51.348 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-05-06T07:37:51.578 INFO:teuthology.kill:Killing Pids: {10209}
2021-05-06T07:37:52.623 INFO:teuthology.kill:Nuking machines: ['gibba014', 'gibba017']
2021-05-06T07:37:53.582 INFO:teuthology.kill:2021-05-06 07:37:53,581.581 INFO:teuthology.nuke:targets:
2021-05-06T07:37:53.583 INFO:teuthology.kill:  gibba014.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1mS/7aNaZP8l75IK7ddtJuOozDa0NWibiywF2tJs+JurQ4IWCx6qRxD902LxGLTLIPYmX/W3lxdcFHCleqXjVB72Pz2+iYCPyAI1TBYwmEirehEsJG580Psoad/XNFQIPLZ9E1P/WVZMiObTFcxZdUSVZCqj+cU66s6hkYnlnoFCCaheVvdusUWpjrwmJT8FNI/6ONYRCWHwMwGSPTDCRUReM0tE8XWC3U2rWSr9+NPvJz8jTvjSStxmo+E2K0UxhuVKyQgWJxTwtQdCva7uU5sxGlrapFlEpLKA2G4clP9NAlURDPi/wkP7HCdL/bgGV2Z/pbefAmimRZ6ES7Qg/s5B7vlxwSi/jr/iA+fNK6PuuIGgKi8TUcWcLpB14ESkEzkwK4KZhUBgOGTEcOmKFqhTVCGJDk4HTICAfv3xpV9HIxPXGf8JY6L8gv2fkI8BEC8L/NONbX30Sk5b/Nq3ceLEmmTPleztSA5ZuSbqRzIG8zePzQY7xt1BE6lSeYgk=
2021-05-06T07:37:53.583 INFO:teuthology.kill:  gibba017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxoWy+ccJpqhOmDMsvGfZgUb7kiwU2UhwI7Gs7EuqHnUQ64divxeU+ciQiCFDkLFNEQBZ995JM1IRznub0fj0C4Kg6U4jEag+BdbkpZ5KP08QzYXLh7uN4GUAmjpcdvg8Vk+KM0BWGR//QmP0pwe1SdCgkzphh1nZ09+ECNK40DQui1c7tz+O6zpx6+VtbA8LARLLohX/wUeD2aRkKhIi+UCCOKdX1nqR23uSYLH26IspTrc2FV6MzaBMZ4K7orAhiI0AOum78YataLRgWjB9bPdzsPhKUeVGvKl1OU4KSr2amQrhCjk492PE3HglhJ0boBvrastkCQQtMLPrCotougN+X5z1CjTX678bWiSv+8X0ZBQ1OeUG7As0GfAe1gVbXve8GWZkhYNaK0dztMMe1th4Vt8Um1KMhiV1xxc+Q2Xicp/Of4fCPSYfj0+591fZAJ3ClIuR4suB0fy2dzKAWOM+Vd/Ydj4Lue0XN7p3poiDBaXqKHFPq3tpyuDgniqs=
2021-05-06T07:37:53.583 INFO:teuthology.kill:2021-05-06 07:37:53,583.583 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-06T07:37:53.586 INFO:teuthology.kill:2021-05-06 07:37:53,586.586 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-06T07:37:53.648 INFO:teuthology.kill:2021-05-06 07:37:53,648.648 INFO:teuthology.orchestra.console:Power off gibba014
2021-05-06T07:37:53.663 INFO:teuthology.kill:2021-05-06 07:37:53,663.663 INFO:teuthology.orchestra.console:Power off gibba017
2021-05-06T07:38:02.026 INFO:teuthology.kill:2021-05-06 07:38:02,025.025 INFO:teuthology.orchestra.console:Power off for gibba014 completed
2021-05-06T07:38:02.033 INFO:teuthology.kill:2021-05-06 07:38:02,033.033 INFO:teuthology.orchestra.console:Power off for gibba017 completed
2021-05-06T07:38:02.256 INFO:teuthology.task.internal:roles: ubuntu@gibba014.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a']
2021-05-06T07:38:02.257 INFO:teuthology.task.internal:roles: ubuntu@gibba017.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']
2021-05-06T07:38:02.257 INFO:teuthology.misc:Compressing logs...
2021-05-06T07:38:02.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-05-06T07:39:02.320 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
    teuth_config.archive_base, job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
    compress_logs(ctx, log_path)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/misc.py", line 1378, in compress_logs
    wait=False,
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in <listcomp>
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/remote.py", line 509, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/run.py", line 434, in run
    transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-05-06T07:39:02.380 INFO:teuthology.kill:No teuthology processes running
2021-05-06T07:39:03.464 INFO:teuthology.kill:Nuking machines: ['gibba014', 'gibba017']
2021-05-06T07:39:04.392 INFO:teuthology.kill:2021-05-06 07:39:04,392.392 INFO:teuthology.nuke:targets:
2021-05-06T07:39:04.393 INFO:teuthology.kill:  gibba014.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1mS/7aNaZP8l75IK7ddtJuOozDa0NWibiywF2tJs+JurQ4IWCx6qRxD902LxGLTLIPYmX/W3lxdcFHCleqXjVB72Pz2+iYCPyAI1TBYwmEirehEsJG580Psoad/XNFQIPLZ9E1P/WVZMiObTFcxZdUSVZCqj+cU66s6hkYnlnoFCCaheVvdusUWpjrwmJT8FNI/6ONYRCWHwMwGSPTDCRUReM0tE8XWC3U2rWSr9+NPvJz8jTvjSStxmo+E2K0UxhuVKyQgWJxTwtQdCva7uU5sxGlrapFlEpLKA2G4clP9NAlURDPi/wkP7HCdL/bgGV2Z/pbefAmimRZ6ES7Qg/s5B7vlxwSi/jr/iA+fNK6PuuIGgKi8TUcWcLpB14ESkEzkwK4KZhUBgOGTEcOmKFqhTVCGJDk4HTICAfv3xpV9HIxPXGf8JY6L8gv2fkI8BEC8L/NONbX30Sk5b/Nq3ceLEmmTPleztSA5ZuSbqRzIG8zePzQY7xt1BE6lSeYgk=
2021-05-06T07:39:04.394 INFO:teuthology.kill:  gibba017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxoWy+ccJpqhOmDMsvGfZgUb7kiwU2UhwI7Gs7EuqHnUQ64divxeU+ciQiCFDkLFNEQBZ995JM1IRznub0fj0C4Kg6U4jEag+BdbkpZ5KP08QzYXLh7uN4GUAmjpcdvg8Vk+KM0BWGR//QmP0pwe1SdCgkzphh1nZ09+ECNK40DQui1c7tz+O6zpx6+VtbA8LARLLohX/wUeD2aRkKhIi+UCCOKdX1nqR23uSYLH26IspTrc2FV6MzaBMZ4K7orAhiI0AOum78YataLRgWjB9bPdzsPhKUeVGvKl1OU4KSr2amQrhCjk492PE3HglhJ0boBvrastkCQQtMLPrCotougN+X5z1CjTX678bWiSv+8X0ZBQ1OeUG7As0GfAe1gVbXve8GWZkhYNaK0dztMMe1th4Vt8Um1KMhiV1xxc+Q2Xicp/Of4fCPSYfj0+591fZAJ3ClIuR4suB0fy2dzKAWOM+Vd/Ydj4Lue0XN7p3poiDBaXqKHFPq3tpyuDgniqs=
2021-05-06T07:39:04.414 INFO:teuthology.kill:2021-05-06 07:39:04,414.414 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-06T07:39:04.417 INFO:teuthology.kill:2021-05-06 07:39:04,417.417 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-06T07:39:04.478 INFO:teuthology.kill:2021-05-06 07:39:04,478.478 INFO:teuthology.orchestra.console:Power off gibba014
2021-05-06T07:39:04.492 INFO:teuthology.kill:2021-05-06 07:39:04,491.491 INFO:teuthology.orchestra.console:Power off gibba017
2021-05-06T07:39:04.627 INFO:teuthology.kill:2021-05-06 07:39:04,626.626 INFO:teuthology.orchestra.console:Power off for gibba014 completed
2021-05-06T07:39:04.639 INFO:teuthology.kill:2021-05-06 07:39:04,639.639 INFO:teuthology.orchestra.console:Power off for gibba017 completed
2021-05-06T07:39:04.770 INFO:teuthology.kill:2021-05-06 07:39:04,769.769 INFO:teuthology.lock.ops:unlocked gibba014.front.sepia.ceph.com
2021-05-06T07:39:04.776 INFO:teuthology.kill:2021-05-06 07:39:04,775.775 ERROR:teuthology.lock.ops:failed to unlock gibba017.front.sepia.ceph.com. reason: 500
2021-05-06T07:41:05.066 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-05-06T07:41:05.195 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-05-06T07:41:05.195 INFO:teuthology.nuke:Checking targets against current locks
2021-05-06T07:41:05.353 INFO:teuthology.nuke:Not nuking gibba014.front.sepia.ceph.com because description doesn't match
2021-05-06T07:41:05.377 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-05-06T07:41:05.450 INFO:teuthology.orchestra.console:Power off gibba017
2021-05-06T07:41:05.647 INFO:teuthology.orchestra.console:Power off for gibba017 completed
2021-05-06T07:41:05.796 INFO:teuthology.lock.ops:unlocked gibba017.front.sepia.ceph.com