2021-02-17T18:22:07.851 INFO:root:teuthology version: 1.1.0-64d5f180 2021-02-17T18:22:07.992 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba028.front.sepia.ceph.com' reimaging 2021-02-17T18:22:07.993 INFO:teuthology.lock.ops:Updating [ubuntu@gibba028.front.sepia.ceph.com]: reset os type and version on server 2021-02-17T18:22:07.993 INFO:teuthology.lock.ops:Updating gibba028.front.sepia.ceph.com on lock server 2021-02-17T18:22:08.014 INFO:teuthology.lock.ops:Node 'ubuntu@gibba028.front.sepia.ceph.com' reimaging is complete 2021-02-17T18:22:08.014 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba038.front.sepia.ceph.com' reimaging 2021-02-17T18:22:08.014 INFO:teuthology.lock.ops:Updating [ubuntu@gibba038.front.sepia.ceph.com]: reset os type and version on server 2021-02-17T18:22:08.015 INFO:teuthology.lock.ops:Updating gibba038.front.sepia.ceph.com on lock server 2021-02-17T18:22:08.036 INFO:teuthology.lock.ops:Node 'ubuntu@gibba038.front.sepia.ceph.com' reimaging is complete 2021-02-17T18:22:08.275 INFO:teuthology.provision.fog.gibba038:Scheduling deploy of centos 8.2 2021-02-17T18:22:08.288 INFO:teuthology.provision.fog.gibba028:Scheduling deploy of centos 8.2 2021-02-17T18:22:08.631 INFO:teuthology.orchestra.console:Power off gibba038 2021-02-17T18:22:08.701 INFO:teuthology.orchestra.console:Power off gibba028 2021-02-17T18:22:17.338 INFO:teuthology.orchestra.console:Power off for gibba038 completed 2021-02-17T18:22:19.474 INFO:teuthology.orchestra.console:Power off for gibba028 completed 2021-02-17T18:22:19.573 INFO:teuthology.orchestra.console:Power on gibba038 2021-02-17T18:22:19.586 INFO:teuthology.orchestra.console:Power on gibba028 2021-02-17T18:22:28.003 INFO:teuthology.orchestra.console:Power on for gibba028 completed 2021-02-17T18:22:28.021 INFO:teuthology.orchestra.console:Power on for gibba038 completed 2021-02-17T18:22:28.105 INFO:teuthology.provision.fog.gibba028:Waiting for deploy to finish 2021-02-17T18:22:28.121 INFO:teuthology.provision.fog.gibba038:Waiting for deploy to finish 2021-02-17T18:24:29.189 ERROR:teuthology.orchestra.connection:Error authenticating with gibba028.front.sepia.ceph.com: Authentication failed. 2021-02-17T18:24:29.196 ERROR:teuthology.orchestra.connection:Error authenticating with gibba038.front.sepia.ceph.com: Authentication failed. 2021-02-17T18:26:51.536 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-17T18:26:56.158 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-02-17T18:26:56.191 INFO:teuthology.provision.fog.gibba038:Node is ready 2021-02-17T18:26:56.202 INFO:teuthology.provision.fog.gibba028:Node is ready 2021-02-17T18:26:56.243 INFO:teuthology.orchestra.run.gibba038.stdout:gibba038.front.sepia.ceph.com 2021-02-17T18:26:56.254 INFO:teuthology.orchestra.run.gibba028.stdout:gibba028.front.sepia.ceph.com 2021-02-17T18:26:56.295 INFO:teuthology.orchestra.run.gibba038.stdout:172.21.2.138 gibba038.front.sepia.ceph.com gibba038 2021-02-17T18:26:56.307 INFO:teuthology.orchestra.run.gibba028.stdout:172.21.2.128 gibba028.front.sepia.ceph.com gibba028 2021-02-17T18:26:56.593 INFO:teuthology.provision.fog.gibba038:Deploy complete! 2021-02-17T18:26:56.609 INFO:teuthology.provision.fog.gibba028:Deploy complete! 2021-02-17T18:26:56.722 INFO:teuthology.lock.ops:Checking gibba028.front.sepia.ceph.com 2021-02-17T18:26:56.722 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-17T18:26:56.743 INFO:teuthology.lock.ops:Checking gibba038.front.sepia.ceph.com 2021-02-17T18:26:56.743 INFO:teuthology.lock.ops:New key found. Updating... 2021-02-17T18:26:56.760 INFO:teuthology.lock.ops:Updating [gibba028.front.sepia.ceph.com]: set os type and version on server 2021-02-17T18:26:56.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-17T18:26:56.959 INFO:teuthology.orchestra.run.gibba028.stdout:x86_64 2021-02-17T18:26:57.011 INFO:teuthology.orchestra.run.gibba028.stdout:NAME="CentOS Linux" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:VERSION="8 (Core)" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:ID="centos" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:ID_LIKE="rhel fedora" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:VERSION_ID="8" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:PLATFORM_ID="platform:el8" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:ANSI_COLOR="0;31" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:HOME_URL="https://www.centos.org/" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-02-17T18:26:57.012 INFO:teuthology.orchestra.run.gibba028.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-17T18:26:57.013 INFO:teuthology.orchestra.run.gibba028.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-17T18:26:57.013 INFO:teuthology.orchestra.run.gibba028.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-17T18:26:57.013 INFO:teuthology.orchestra.run.gibba028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-17T18:26:57.013 INFO:teuthology.orchestra.run.gibba028.stdout: 2021-02-17T18:26:57.014 INFO:teuthology.lock.ops:Updating gibba028.front.sepia.ceph.com on lock server 2021-02-17T18:26:57.029 INFO:teuthology.lock.ops:Updating [gibba038.front.sepia.ceph.com]: set os type and version on server 2021-02-17T18:26:57.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-17T18:26:57.223 INFO:teuthology.orchestra.run.gibba038.stdout:x86_64 2021-02-17T18:26:57.276 INFO:teuthology.orchestra.run.gibba038.stdout:NAME="CentOS Linux" 2021-02-17T18:26:57.276 INFO:teuthology.orchestra.run.gibba038.stdout:VERSION="8 (Core)" 2021-02-17T18:26:57.277 INFO:teuthology.orchestra.run.gibba038.stdout:ID="centos" 2021-02-17T18:26:57.277 INFO:teuthology.orchestra.run.gibba038.stdout:ID_LIKE="rhel fedora" 2021-02-17T18:26:57.277 INFO:teuthology.orchestra.run.gibba038.stdout:VERSION_ID="8" 2021-02-17T18:26:57.277 INFO:teuthology.orchestra.run.gibba038.stdout:PLATFORM_ID="platform:el8" 2021-02-17T18:26:57.277 INFO:teuthology.orchestra.run.gibba038.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-02-17T18:26:57.277 INFO:teuthology.orchestra.run.gibba038.stdout:ANSI_COLOR="0;31" 2021-02-17T18:26:57.278 INFO:teuthology.orchestra.run.gibba038.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-02-17T18:26:57.278 INFO:teuthology.orchestra.run.gibba038.stdout:HOME_URL="https://www.centos.org/" 2021-02-17T18:26:57.278 INFO:teuthology.orchestra.run.gibba038.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-02-17T18:26:57.278 INFO:teuthology.orchestra.run.gibba038.stdout: 2021-02-17T18:26:57.278 INFO:teuthology.orchestra.run.gibba038.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-02-17T18:26:57.279 INFO:teuthology.orchestra.run.gibba038.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-02-17T18:26:57.279 INFO:teuthology.orchestra.run.gibba038.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-02-17T18:26:57.279 INFO:teuthology.orchestra.run.gibba038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-02-17T18:26:57.279 INFO:teuthology.orchestra.run.gibba038.stdout: 2021-02-17T18:26:57.280 INFO:teuthology.lock.ops:Updating gibba038.front.sepia.ceph.com on lock server 2021-02-17T18:26:57.354 INFO:teuthology.dispatcher.supervisor:Running job 5886297 2021-02-17T18:26:57.354 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-02-16_01:01:03-crimson-rados-master-distro-basic-gibba/5886297 --name teuthology-2021-02-16_01:01:03-crimson-rados-master-distro-basic-gibba --description crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects} -- /home/teuthworker/archive/teuthology-2021-02-16_01:01:03-crimson-rados-master-distro-basic-gibba/5886297/orig.config.yaml 2021-02-17T18:26:57.358 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-02-16_01:01:03-crimson-rados-master-distro-basic-gibba/5886297 2021-02-17T18:26:57.359 INFO:teuthology.dispatcher.supervisor:Job PID: 2419 2021-02-17T18:26:57.359 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-02-18T06:27:12.960 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-02-18T06:27:18.870 INFO:teuthology.kill:Killing Pids: {2419} 2021-02-18T06:27:19.853 INFO:teuthology.kill:Nuking machines: ['gibba028', 'gibba038'] 2021-02-18T06:27:20.696 INFO:teuthology.kill:2021-02-18 06:27:20,696.696 INFO:teuthology.nuke:targets: 2021-02-18T06:27:20.697 INFO:teuthology.kill: gibba028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDX9at5QvOUl5RkNrQCRjXKb6nT4/a/rAcNgdqEntcyAKukx2xQwMdpp8+74IPGE9h09sMK40d7EQUxcVB17RYGYOac9btRYgDg/BN5/HkLfY5vNc6dY1DmLMTDRv/2twR0Y3bthfR28FMw9mhapUJ6kdPWe1I76vw12J/YOHdj/1CCOwMr+JfrCg6X7+sV3BO6lwqm+l/7/PJ4t0vgvKGzdlgGzp60+gF8/pOg75FNz6dQv8kwYtaeklYzcR6Hv26wFODJF+LmpfS56so+LogBT/Yusf+PQ/KvZ10qjj5Qt64A+3d2v0IFE68B8fRuTZ0gP10dtF4FqXk5G4UeN/FcRPZFAM/Qd25G21DZUmnMHluEKixYkI1fOPGdWBmm7aTsAEsYGOFkLaNiWVaGm2/0RFH2lIeN8f0cGzTvkkNg+Qgw6d7HIt5gzKtYhnWSiGUrz9pWPhIhAJ0997/WlD8F1vNMwMNOEY46Pz2yr2NF/VNYtb0NsOUZzIPVfb25HLc= 2021-02-18T06:27:20.697 INFO:teuthology.kill: gibba038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMmNxOLdBLObZrun6GwjSv7FYk3xULM9VRlmYz3qkXFkYJw0kMdMeTt604BCuCJNKAN0GC5xe17/0ny+GFA/eCw+GV049gR2oFYnAPEwa6+WpoQYnom2bK7zrbMDuH3dlSID+sA7vus6tpf6aKyVQZIX8jM0IBIwoCXXtRRC9jCr2JqzXm+gQaZkDgGzi7fZb8Sfpk9kIj/b0GtfWqEpsJnPqQTQgkcLSHukZJA+yQXpVWFHYWYlt5j91HxDR5qZeGVigmsNNvmqGOZsYBTRIcrESM+Vx49sGT2V8o4/VAYC4gq3WHSrdR73/YdzyEo3EUO63hXj7ZzGMYIP+eImKZLW9ZcTaMINPe/COke+hZFk9PwRFSlhjnjfGBmiU59cQXCsiIoueinl7VUByLeg53Z/Pm+P0Nv++xR3d3g/J/r3RkXKVcCmnet+QKzUnS5Ktlc0XvBiaoahff2MDPifAc7VFSjqKVPONU2rxmoyhUI4ZdOyQH3oDS1LTpLDgLyQ0= 2021-02-18T06:27:20.697 INFO:teuthology.kill:2021-02-18 06:27:20,696.696 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-18T06:27:20.702 INFO:teuthology.kill:2021-02-18 06:27:20,702.702 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-18T06:27:20.762 INFO:teuthology.kill:2021-02-18 06:27:20,762.762 INFO:teuthology.orchestra.console:Power off gibba038 2021-02-18T06:27:20.778 INFO:teuthology.kill:2021-02-18 06:27:20,777.777 INFO:teuthology.orchestra.console:Power off gibba028 2021-02-18T06:27:29.137 INFO:teuthology.kill:2021-02-18 06:27:29,137.137 INFO:teuthology.orchestra.console:Power off for gibba038 completed 2021-02-18T06:27:33.264 INFO:teuthology.kill:2021-02-18 06:27:33,264.264 INFO:teuthology.orchestra.console:Power off for gibba028 completed 2021-02-18T06:27:33.492 INFO:teuthology.task.internal:roles: ubuntu@gibba028.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2021-02-18T06:27:33.492 INFO:teuthology.task.internal:roles: ubuntu@gibba038.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-02-18T06:27:33.492 INFO:teuthology.misc:Compressing logs... 2021-02-18T06:27:33.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-02-18T06:28:33.496 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/misc.py", line 1372, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_64d5f18051c2fa3ad001b1c48f945134f60a8f76/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-02-18T06:28:33.557 INFO:teuthology.kill:No teuthology processes running 2021-02-18T06:28:34.562 INFO:teuthology.kill:Nuking machines: ['gibba028', 'gibba038'] 2021-02-18T06:28:35.346 INFO:teuthology.kill:2021-02-18 06:28:35,346.346 INFO:teuthology.nuke:targets: 2021-02-18T06:28:35.347 INFO:teuthology.kill: gibba028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDX9at5QvOUl5RkNrQCRjXKb6nT4/a/rAcNgdqEntcyAKukx2xQwMdpp8+74IPGE9h09sMK40d7EQUxcVB17RYGYOac9btRYgDg/BN5/HkLfY5vNc6dY1DmLMTDRv/2twR0Y3bthfR28FMw9mhapUJ6kdPWe1I76vw12J/YOHdj/1CCOwMr+JfrCg6X7+sV3BO6lwqm+l/7/PJ4t0vgvKGzdlgGzp60+gF8/pOg75FNz6dQv8kwYtaeklYzcR6Hv26wFODJF+LmpfS56so+LogBT/Yusf+PQ/KvZ10qjj5Qt64A+3d2v0IFE68B8fRuTZ0gP10dtF4FqXk5G4UeN/FcRPZFAM/Qd25G21DZUmnMHluEKixYkI1fOPGdWBmm7aTsAEsYGOFkLaNiWVaGm2/0RFH2lIeN8f0cGzTvkkNg+Qgw6d7HIt5gzKtYhnWSiGUrz9pWPhIhAJ0997/WlD8F1vNMwMNOEY46Pz2yr2NF/VNYtb0NsOUZzIPVfb25HLc= 2021-02-18T06:28:35.347 INFO:teuthology.kill: gibba038.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMmNxOLdBLObZrun6GwjSv7FYk3xULM9VRlmYz3qkXFkYJw0kMdMeTt604BCuCJNKAN0GC5xe17/0ny+GFA/eCw+GV049gR2oFYnAPEwa6+WpoQYnom2bK7zrbMDuH3dlSID+sA7vus6tpf6aKyVQZIX8jM0IBIwoCXXtRRC9jCr2JqzXm+gQaZkDgGzi7fZb8Sfpk9kIj/b0GtfWqEpsJnPqQTQgkcLSHukZJA+yQXpVWFHYWYlt5j91HxDR5qZeGVigmsNNvmqGOZsYBTRIcrESM+Vx49sGT2V8o4/VAYC4gq3WHSrdR73/YdzyEo3EUO63hXj7ZzGMYIP+eImKZLW9ZcTaMINPe/COke+hZFk9PwRFSlhjnjfGBmiU59cQXCsiIoueinl7VUByLeg53Z/Pm+P0Nv++xR3d3g/J/r3RkXKVcCmnet+QKzUnS5Ktlc0XvBiaoahff2MDPifAc7VFSjqKVPONU2rxmoyhUI4ZdOyQH3oDS1LTpLDgLyQ0= 2021-02-18T06:28:35.369 INFO:teuthology.kill:2021-02-18 06:28:35,369.369 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-18T06:28:35.372 INFO:teuthology.kill:2021-02-18 06:28:35,372.372 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-02-18T06:28:35.429 INFO:teuthology.kill:2021-02-18 06:28:35,428.428 INFO:teuthology.orchestra.console:Power off gibba028 2021-02-18T06:28:35.445 INFO:teuthology.kill:2021-02-18 06:28:35,444.444 INFO:teuthology.orchestra.console:Power off gibba038 2021-02-18T06:28:35.580 INFO:teuthology.kill:2021-02-18 06:28:35,580.580 INFO:teuthology.orchestra.console:Power off for gibba028 completed 2021-02-18T06:28:35.595 INFO:teuthology.kill:2021-02-18 06:28:35,595.595 INFO:teuthology.orchestra.console:Power off for gibba038 completed 2021-02-18T06:28:35.722 INFO:teuthology.kill:2021-02-18 06:28:35,721.721 INFO:teuthology.lock.ops:unlocked gibba028.front.sepia.ceph.com 2021-02-18T06:28:35.734 INFO:teuthology.kill:2021-02-18 06:28:35,734.734 INFO:teuthology.lock.ops:unlocked gibba038.front.sepia.ceph.com 2021-02-18T06:30:35.920 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-02-18T06:30:36.035 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-02-18T06:30:36.035 INFO:teuthology.nuke:Checking targets against current locks 2021-02-18T06:30:36.110 INFO:teuthology.nuke:Not nuking gibba038.front.sepia.ceph.com because description doesn't match 2021-02-18T06:30:36.110 INFO:teuthology.nuke:Not nuking gibba028.front.sepia.ceph.com because description doesn't match