2021-02-16T09:42:30.497 INFO:root:teuthology version: 1.1.0-071a4df2 2021-02-16T09:42:30.603 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba023.front.sepia.ceph.com' reimaging 2021-02-16T09:42:30.604 INFO:teuthology.lock.ops:Updating [ubuntu@gibba023.front.sepia.ceph.com]: reset os type and version on server 2021-02-16T09:42:30.604 INFO:teuthology.lock.ops:Updating gibba023.front.sepia.ceph.com on lock server 2021-02-16T09:42:30.617 INFO:teuthology.lock.ops:Node 'ubuntu@gibba023.front.sepia.ceph.com' reimaging is complete 2021-02-16T09:42:30.618 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba024.front.sepia.ceph.com' reimaging 2021-02-16T09:42:30.618 INFO:teuthology.lock.ops:Updating [ubuntu@gibba024.front.sepia.ceph.com]: reset os type and version on server 2021-02-16T09:42:30.618 INFO:teuthology.lock.ops:Updating gibba024.front.sepia.ceph.com on lock server 2021-02-16T09:42:30.630 INFO:teuthology.lock.ops:Node 'ubuntu@gibba024.front.sepia.ceph.com' reimaging is complete 2021-02-16T09:42:30.812 INFO:teuthology.provision.fog.gibba023:Scheduling deploy of rhel 8.3 2021-02-16T09:42:30.819 INFO:teuthology.provision.fog.gibba024:Scheduling deploy of rhel 8.3 2021-02-16T09:42:31.183 INFO:teuthology.orchestra.console:Power off gibba024 2021-02-16T09:42:31.196 INFO:teuthology.orchestra.console:Power off gibba023 2021-02-16T09:42:39.575 INFO:teuthology.orchestra.console:Power off for gibba023 completed 2021-02-16T09:42:39.676 INFO:teuthology.orchestra.console:Power on gibba023 2021-02-16T09:42:43.672 INFO:teuthology.orchestra.console:Power off for gibba024 completed 2021-02-16T09:42:43.773 INFO:teuthology.orchestra.console:Power on gibba024 2021-02-16T09:42:48.045 INFO:teuthology.orchestra.console:Power on for gibba023 completed 2021-02-16T09:42:48.146 INFO:teuthology.provision.fog.gibba023:Waiting for deploy to finish 2021-02-16T09:42:52.231 INFO:teuthology.orchestra.console:Power on for gibba024 completed 2021-02-16T09:42:52.333 INFO:teuthology.provision.fog.gibba024:Waiting for deploy to finish 2021-02-16T09:46:47.411 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-16T09:46:53.180 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-16T09:46:57.876 INFO:teuthology.provision.fog.gibba023:Node is ready 2021-02-16T09:46:57.889 INFO:teuthology.orchestra.run.gibba023.stdout:gibba029 2021-02-16T09:46:57.942 INFO:teuthology.orchestra.run.gibba023.stdout:172.21.2.129 gibba029.front.sepia.ceph.com gibba029 2021-02-16T09:46:58.262 INFO:teuthology.provision.fog.gibba023:Deploy complete! 2021-02-16T09:47:03.454 INFO:teuthology.provision.fog.gibba024:Node is ready 2021-02-16T09:47:03.466 INFO:teuthology.orchestra.run.gibba024.stdout:gibba024.front.sepia.ceph.com 2021-02-16T09:47:03.517 INFO:teuthology.orchestra.run.gibba024.stdout:172.21.2.124 gibba024.front.sepia.ceph.com gibba024 2021-02-16T09:47:03.832 INFO:teuthology.provision.fog.gibba024:Deploy complete! 2021-02-16T09:47:03.954 INFO:teuthology.lock.ops:Checking gibba023.front.sepia.ceph.com 2021-02-16T09:47:03.954 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-16T09:47:03.971 INFO:teuthology.lock.ops:Checking gibba024.front.sepia.ceph.com 2021-02-16T09:47:03.971 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-16T09:47:03.985 INFO:teuthology.lock.ops:Updating [gibba023.front.sepia.ceph.com]: set os type and version on server 2021-02-16T09:47:03.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-16T09:47:04.228 INFO:teuthology.orchestra.run.gibba023.stdout:x86_64 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:NAME="Red Hat Enterprise Linux" 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:VERSION="8.3 (Ootpa)" 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:ID="rhel" 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:ID_LIKE="fedora" 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:VERSION_ID="8.3" 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:PLATFORM_ID="platform:el8" 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:ANSI_COLOR="0;31" 2021-02-16T09:47:04.280 INFO:teuthology.orchestra.run.gibba023.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-02-16T09:47:04.281 INFO:teuthology.orchestra.run.gibba023.stdout:HOME_URL="https://www.redhat.com/" 2021-02-16T09:47:04.281 INFO:teuthology.orchestra.run.gibba023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-16T09:47:04.281 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-02-16T09:47:04.281 INFO:teuthology.orchestra.run.gibba023.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-02-16T09:47:04.281 INFO:teuthology.orchestra.run.gibba023.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-02-16T09:47:04.281 INFO:teuthology.orchestra.run.gibba023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-16T09:47:04.281 INFO:teuthology.orchestra.run.gibba023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-02-16T09:47:04.282 INFO:teuthology.lock.ops:Updating gibba023.front.sepia.ceph.com on lock server 2021-02-16T09:47:04.297 INFO:teuthology.lock.ops:Updating [gibba024.front.sepia.ceph.com]: set os type and version on server 2021-02-16T09:47:04.297 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-16T09:47:04.545 INFO:teuthology.orchestra.run.gibba024.stdout:x86_64 2021-02-16T09:47:04.596 INFO:teuthology.orchestra.run.gibba024.stdout:NAME="Red Hat Enterprise Linux" 2021-02-16T09:47:04.596 INFO:teuthology.orchestra.run.gibba024.stdout:VERSION="8.3 (Ootpa)" 2021-02-16T09:47:04.597 INFO:teuthology.orchestra.run.gibba024.stdout:ID="rhel" 2021-02-16T09:47:04.597 INFO:teuthology.orchestra.run.gibba024.stdout:ID_LIKE="fedora" 2021-02-16T09:47:04.597 INFO:teuthology.orchestra.run.gibba024.stdout:VERSION_ID="8.3" 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout:PLATFORM_ID="platform:el8" 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.3 (Ootpa)" 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout:ANSI_COLOR="0;31" 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.3:GA" 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout:HOME_URL="https://www.redhat.com/" 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout: 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2021-02-16T09:47:04.598 INFO:teuthology.orchestra.run.gibba024.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.3 2021-02-16T09:47:04.599 INFO:teuthology.orchestra.run.gibba024.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2021-02-16T09:47:04.602 INFO:teuthology.orchestra.run.gibba024.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.3" 2021-02-16T09:47:04.603 INFO:teuthology.lock.ops:Updating gibba024.front.sepia.ceph.com on lock server 2021-02-16T09:47:04.683 INFO:teuthology.dispatcher.supervisor:Running job 5887517 2021-02-16T09:47:04.684 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/virtualenv/bin/teuthology -v --owner vshankar --archive /home/teuthworker/archive/vshankar-2021-02-16_09:38:04-fs-wip-vshankar-testing-20210215-050616-distro-basic-gibba/5887517 --name vshankar-2021-02-16_09:38:04-fs-wip-vshankar-testing-20210215-050616-distro-basic-gibba --description fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{faked-ino frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cfuse_workunit_suites_fsstress} -- /home/teuthworker/archive/vshankar-2021-02-16_09:38:04-fs-wip-vshankar-testing-20210215-050616-distro-basic-gibba/5887517/orig.config.yaml 2021-02-16T09:47:04.687 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2021-02-16_09:38:04-fs-wip-vshankar-testing-20210215-050616-distro-basic-gibba/5887517 2021-02-16T09:47:04.687 INFO:teuthology.dispatcher.supervisor:Job PID: 4108 2021-02-16T09:47:04.687 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-16T21:47:21.244 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-16T21:47:22.132 INFO:teuthology.kill:Killing Pids: {4108} 2021-02-16T21:47:23.927 INFO:teuthology.kill:Nuking machines: ['gibba023', 'gibba024'] 2021-02-16T21:47:24.739 INFO:teuthology.kill:2021-02-16 21:47:24,738.738 INFO:teuthology.nuke:targets: 2021-02-16T21:47:24.739 INFO:teuthology.kill: gibba023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDqxrDsDBN04scw9OMcfCWUJgRgB2v4tKJEneBQFTCCAgWAL9hT0bwwKMcSv+7mPU5m6c4VwfRIW3IMViiYYgzG1MxvyA/GZ1AGdORk5mRvyWZloS0RMzGaLU6VWEJkbNdqhEW2Zc8ola0eCRbwiyQtUu1n25r7xGzl3STC4Fv1mz+9EBgBJvUWYDJ2YKbRHVuhwxaYPWNoc8J2RlXK4zIzpKqJkDGn0g58zBLx6hM08MZoh5NWgd9eLuyY8nqzMCuW3h1kD8eoz1Ahhzceu1tLTu9MG+um8uV8klyvrW6oBIxO7BgnaHavtWKjnt0wixPIw+wLlR2W5dX02JUe/Z20vfEAv/24R8JyloGmWUKZUhRXM1P94Hdm2CmMrtdh7P6wN0dEUtJs4D9dfktWjteUKFJKxXJrrEt5n0oA8ZFgX5yFidQeez/EsJz13GMI/nKAAOEYN8/Mf17HsUHKfVFA60angeryxmoB8x1QuejgbkHEJhcuUl4AkpPZAWdVbYc= 2021-02-16T21:47:24.739 INFO:teuthology.kill: gibba024.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8kE5293TOyMFdWVQE4gnryjiq/Z502Bd99Z6VBcli/HZi5rHQVErt0m2WWI/SWMNbLgYEy1X6IDVIuchEifofrJbFCmcjNN0MCzGZSmC64Q4VQztAXC6RIe3OGL5he7DJHcdy6+IkXlgmbptSor32wAZzJTs1Z1lxhQIBZlGSgsY9WhzBma+mJ6pngNLy8RtV1ky7ub/0tALqNI77ZngXabIaaKmoEJqBmdiaW5pKlnbejt5aGSQ2HLaK6kExXbb6Ad6PlqA97wkSCuhUcpsxlayDAGzjKWiHgsoRk9W0b6ON56j2jppShPVSTBrKan/dcxFcBDCXmPIcf/y+UYysTET7Zy8HGl85tX5SJySkh1sfbSbeF4u5DifRBCKsdjZNmSREEN8MCmP0UPlALtrJqC6VMBoIbfp46F29/HyrZE+eAEalL5uommtqZ9lL0h0j2mZ9wb0q34USakYGIhHE3jvFyswp7Ml0/LozqhymsoXfMjCRmBDl/5TDXPNOaRs= 2021-02-16T21:47:24.739 INFO:teuthology.kill:2021-02-16 21:47:24,739.739 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-16T21:47:24.745 INFO:teuthology.kill:2021-02-16 21:47:24,745.745 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-16T21:47:24.806 INFO:teuthology.kill:2021-02-16 21:47:24,805.805 INFO:teuthology.orchestra.console:Power off gibba023 2021-02-16T21:47:24.821 INFO:teuthology.kill:2021-02-16 21:47:24,821.821 INFO:teuthology.orchestra.console:Power off gibba024 2021-02-16T21:47:33.208 INFO:teuthology.kill:2021-02-16 21:47:33,207.207 INFO:teuthology.orchestra.console:Power off for gibba023 completed 2021-02-16T21:47:33.305 INFO:teuthology.kill:2021-02-16 21:47:33,213.213 INFO:teuthology.orchestra.console:Power off for gibba024 completed 2021-02-16T21:47:33.477 INFO:teuthology.task.internal:roles: ubuntu@gibba023.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-02-16T21:47:33.478 INFO:teuthology.task.internal:roles: ubuntu@gibba024.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-02-16T21:47:33.481 INFO:teuthology.misc:Compressing logs... 2021-02-16T21:47:33.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-16T21:48:33.588 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/misc.py", line 1372, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_071a4df2ce463088af781721173908ae84c1e2bd/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-16T21:48:33.695 INFO:teuthology.kill:No teuthology processes running 2021-02-16T21:48:35.156 INFO:teuthology.kill:Nuking machines: ['gibba023', 'gibba024'] 2021-02-16T21:48:35.926 INFO:teuthology.kill:2021-02-16 21:48:35,926.926 INFO:teuthology.nuke:targets: 2021-02-16T21:48:35.927 INFO:teuthology.kill: gibba023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDqxrDsDBN04scw9OMcfCWUJgRgB2v4tKJEneBQFTCCAgWAL9hT0bwwKMcSv+7mPU5m6c4VwfRIW3IMViiYYgzG1MxvyA/GZ1AGdORk5mRvyWZloS0RMzGaLU6VWEJkbNdqhEW2Zc8ola0eCRbwiyQtUu1n25r7xGzl3STC4Fv1mz+9EBgBJvUWYDJ2YKbRHVuhwxaYPWNoc8J2RlXK4zIzpKqJkDGn0g58zBLx6hM08MZoh5NWgd9eLuyY8nqzMCuW3h1kD8eoz1Ahhzceu1tLTu9MG+um8uV8klyvrW6oBIxO7BgnaHavtWKjnt0wixPIw+wLlR2W5dX02JUe/Z20vfEAv/24R8JyloGmWUKZUhRXM1P94Hdm2CmMrtdh7P6wN0dEUtJs4D9dfktWjteUKFJKxXJrrEt5n0oA8ZFgX5yFidQeez/EsJz13GMI/nKAAOEYN8/Mf17HsUHKfVFA60angeryxmoB8x1QuejgbkHEJhcuUl4AkpPZAWdVbYc= 2021-02-16T21:48:35.927 INFO:teuthology.kill: gibba024.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8kE5293TOyMFdWVQE4gnryjiq/Z502Bd99Z6VBcli/HZi5rHQVErt0m2WWI/SWMNbLgYEy1X6IDVIuchEifofrJbFCmcjNN0MCzGZSmC64Q4VQztAXC6RIe3OGL5he7DJHcdy6+IkXlgmbptSor32wAZzJTs1Z1lxhQIBZlGSgsY9WhzBma+mJ6pngNLy8RtV1ky7ub/0tALqNI77ZngXabIaaKmoEJqBmdiaW5pKlnbejt5aGSQ2HLaK6kExXbb6Ad6PlqA97wkSCuhUcpsxlayDAGzjKWiHgsoRk9W0b6ON56j2jppShPVSTBrKan/dcxFcBDCXmPIcf/y+UYysTET7Zy8HGl85tX5SJySkh1sfbSbeF4u5DifRBCKsdjZNmSREEN8MCmP0UPlALtrJqC6VMBoIbfp46F29/HyrZE+eAEalL5uommtqZ9lL0h0j2mZ9wb0q34USakYGIhHE3jvFyswp7Ml0/LozqhymsoXfMjCRmBDl/5TDXPNOaRs= 2021-02-16T21:48:35.950 INFO:teuthology.kill:2021-02-16 21:48:35,950.950 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-16T21:48:35.953 INFO:teuthology.kill:2021-02-16 21:48:35,953.953 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-16T21:48:36.009 INFO:teuthology.kill:2021-02-16 21:48:36,009.009 INFO:teuthology.orchestra.console:Power off gibba023 2021-02-16T21:48:36.022 INFO:teuthology.kill:2021-02-16 21:48:36,022.022 INFO:teuthology.orchestra.console:Power off gibba024 2021-02-16T21:48:36.158 INFO:teuthology.kill:2021-02-16 21:48:36,158.158 INFO:teuthology.orchestra.console:Power off for gibba023 completed 2021-02-16T21:48:36.173 INFO:teuthology.kill:2021-02-16 21:48:36,172.172 INFO:teuthology.orchestra.console:Power off for gibba024 completed 2021-02-16T21:48:36.307 INFO:teuthology.kill:2021-02-16 21:48:36,306.306 INFO:teuthology.lock.ops:unlocked gibba023.front.sepia.ceph.com 2021-02-16T21:48:36.313 INFO:teuthology.kill:2021-02-16 21:48:36,312.312 INFO:teuthology.lock.ops:unlocked gibba024.front.sepia.ceph.com 2021-02-16T21:50:36.485 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-16T21:50:36.595 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-16T21:50:36.597 INFO:teuthology.nuke:Checking targets against current locks 2021-02-16T21:50:36.823 INFO:teuthology.nuke:Not nuking gibba023.front.sepia.ceph.com because description doesn't match 2021-02-16T21:50:36.823 INFO:teuthology.nuke:Not nuking gibba024.front.sepia.ceph.com because description doesn't match