2021-03-08T07:36:13.496 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-08T07:36:13.594 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba027.front.sepia.ceph.com' reimaging 2021-03-08T07:36:13.595 INFO:teuthology.lock.ops:Updating [ubuntu@gibba027.front.sepia.ceph.com]: reset os type and version on server 2021-03-08T07:36:13.595 INFO:teuthology.lock.ops:Updating gibba027.front.sepia.ceph.com on lock server 2021-03-08T07:36:13.610 INFO:teuthology.lock.ops:Node 'ubuntu@gibba027.front.sepia.ceph.com' reimaging is complete 2021-03-08T07:36:13.610 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba036.front.sepia.ceph.com' reimaging 2021-03-08T07:36:13.611 INFO:teuthology.lock.ops:Updating [ubuntu@gibba036.front.sepia.ceph.com]: reset os type and version on server 2021-03-08T07:36:13.611 INFO:teuthology.lock.ops:Updating gibba036.front.sepia.ceph.com on lock server 2021-03-08T07:36:13.626 INFO:teuthology.lock.ops:Node 'ubuntu@gibba036.front.sepia.ceph.com' reimaging is complete 2021-03-08T07:36:13.826 INFO:teuthology.provision.fog.gibba027:Scheduling deploy of centos 8.2 2021-03-08T07:36:13.829 INFO:teuthology.provision.fog.gibba036:Scheduling deploy of centos 8.2 2021-03-08T07:36:14.342 INFO:teuthology.orchestra.console:Power off gibba036 2021-03-08T07:36:14.422 INFO:teuthology.orchestra.console:Power off gibba027 2021-03-08T07:36:22.764 INFO:teuthology.orchestra.console:Power off for gibba036 completed 2021-03-08T07:36:22.827 INFO:teuthology.orchestra.console:Power off for gibba027 completed 2021-03-08T07:36:22.867 INFO:teuthology.orchestra.console:Power on gibba036 2021-03-08T07:36:22.928 INFO:teuthology.orchestra.console:Power on gibba027 2021-03-08T07:36:31.268 INFO:teuthology.orchestra.console:Power on for gibba036 completed 2021-03-08T07:36:31.305 INFO:teuthology.orchestra.console:Power on for gibba027 completed 2021-03-08T07:36:31.370 INFO:teuthology.provision.fog.gibba036:Waiting for deploy to finish 2021-03-08T07:36:31.407 INFO:teuthology.provision.fog.gibba027:Waiting for deploy to finish 2021-03-08T07:40:35.034 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-08T07:40:44.860 INFO:teuthology.provision.fog.gibba027:Node is ready 2021-03-08T07:40:44.874 INFO:teuthology.orchestra.run.gibba027.stdout:gibba027.front.sepia.ceph.com 2021-03-08T07:40:44.928 INFO:teuthology.orchestra.run.gibba027.stdout:172.21.2.127 gibba027.front.sepia.ceph.com gibba027 2021-03-08T07:40:45.258 INFO:teuthology.provision.fog.gibba027:Deploy complete! 2021-03-08T07:40:57.475 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-08T07:40:57.697 INFO:teuthology.provision.fog.gibba036:Node is ready 2021-03-08T07:40:57.708 INFO:teuthology.orchestra.run.gibba036.stdout:gibba036.front.sepia.ceph.com 2021-03-08T07:40:57.761 INFO:teuthology.orchestra.run.gibba036.stdout:172.21.2.136 gibba036.front.sepia.ceph.com gibba036 2021-03-08T07:40:58.058 INFO:teuthology.provision.fog.gibba036:Deploy complete! 2021-03-08T07:40:58.193 INFO:teuthology.lock.ops:Checking gibba027.front.sepia.ceph.com 2021-03-08T07:40:58.197 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-08T07:40:58.215 INFO:teuthology.lock.ops:Checking gibba036.front.sepia.ceph.com 2021-03-08T07:40:58.216 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-08T07:40:58.231 INFO:teuthology.lock.ops:Updating [gibba027.front.sepia.ceph.com]: set os type and version on server 2021-03-08T07:40:58.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-08T07:40:58.424 INFO:teuthology.orchestra.run.gibba027.stdout:x86_64 2021-03-08T07:40:58.477 INFO:teuthology.orchestra.run.gibba027.stdout:NAME="CentOS Linux" 2021-03-08T07:40:58.478 INFO:teuthology.orchestra.run.gibba027.stdout:VERSION="8 (Core)" 2021-03-08T07:40:58.478 INFO:teuthology.orchestra.run.gibba027.stdout:ID="centos" 2021-03-08T07:40:58.478 INFO:teuthology.orchestra.run.gibba027.stdout:ID_LIKE="rhel fedora" 2021-03-08T07:40:58.479 INFO:teuthology.orchestra.run.gibba027.stdout:VERSION_ID="8" 2021-03-08T07:40:58.479 INFO:teuthology.orchestra.run.gibba027.stdout:PLATFORM_ID="platform:el8" 2021-03-08T07:40:58.480 INFO:teuthology.orchestra.run.gibba027.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-03-08T07:40:58.480 INFO:teuthology.orchestra.run.gibba027.stdout:ANSI_COLOR="0;31" 2021-03-08T07:40:58.480 INFO:teuthology.orchestra.run.gibba027.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-03-08T07:40:58.480 INFO:teuthology.orchestra.run.gibba027.stdout:HOME_URL="https://www.centos.org/" 2021-03-08T07:40:58.480 INFO:teuthology.orchestra.run.gibba027.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-03-08T07:40:58.481 INFO:teuthology.orchestra.run.gibba027.stdout: 2021-03-08T07:40:58.481 INFO:teuthology.orchestra.run.gibba027.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-03-08T07:40:58.481 INFO:teuthology.orchestra.run.gibba027.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-03-08T07:40:58.481 INFO:teuthology.orchestra.run.gibba027.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-03-08T07:40:58.482 INFO:teuthology.orchestra.run.gibba027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-03-08T07:40:58.482 INFO:teuthology.orchestra.run.gibba027.stdout: 2021-03-08T07:40:58.483 INFO:teuthology.lock.ops:Updating gibba027.front.sepia.ceph.com on lock server 2021-03-08T07:40:58.502 INFO:teuthology.lock.ops:Updating [gibba036.front.sepia.ceph.com]: set os type and version on server 2021-03-08T07:40:58.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-08T07:40:58.738 INFO:teuthology.orchestra.run.gibba036.stdout:x86_64 2021-03-08T07:40:58.792 INFO:teuthology.orchestra.run.gibba036.stdout:NAME="CentOS Linux" 2021-03-08T07:40:58.792 INFO:teuthology.orchestra.run.gibba036.stdout:VERSION="8 (Core)" 2021-03-08T07:40:58.793 INFO:teuthology.orchestra.run.gibba036.stdout:ID="centos" 2021-03-08T07:40:58.793 INFO:teuthology.orchestra.run.gibba036.stdout:ID_LIKE="rhel fedora" 2021-03-08T07:40:58.793 INFO:teuthology.orchestra.run.gibba036.stdout:VERSION_ID="8" 2021-03-08T07:40:58.793 INFO:teuthology.orchestra.run.gibba036.stdout:PLATFORM_ID="platform:el8" 2021-03-08T07:40:58.794 INFO:teuthology.orchestra.run.gibba036.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-03-08T07:40:58.794 INFO:teuthology.orchestra.run.gibba036.stdout:ANSI_COLOR="0;31" 2021-03-08T07:40:58.794 INFO:teuthology.orchestra.run.gibba036.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-03-08T07:40:58.794 INFO:teuthology.orchestra.run.gibba036.stdout:HOME_URL="https://www.centos.org/" 2021-03-08T07:40:58.794 INFO:teuthology.orchestra.run.gibba036.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-03-08T07:40:58.795 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-03-08T07:40:58.795 INFO:teuthology.orchestra.run.gibba036.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-03-08T07:40:58.795 INFO:teuthology.orchestra.run.gibba036.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-03-08T07:40:58.795 INFO:teuthology.orchestra.run.gibba036.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-03-08T07:40:58.797 INFO:teuthology.orchestra.run.gibba036.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-03-08T07:40:58.798 INFO:teuthology.orchestra.run.gibba036.stdout: 2021-03-08T07:40:58.799 INFO:teuthology.lock.ops:Updating gibba036.front.sepia.ceph.com on lock server 2021-03-08T07:40:58.869 INFO:teuthology.dispatcher.supervisor:Running job 5942880 2021-03-08T07:40:58.869 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-03-07_01:01:03-crimson-rados-master-distro-basic-gibba/5942880 --name teuthology-2021-03-07_01:01:03-crimson-rados-master-distro-basic-gibba --description crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} centos_latest clusters/{fixed-2} deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/small-objects} -- /home/teuthworker/archive/teuthology-2021-03-07_01:01:03-crimson-rados-master-distro-basic-gibba/5942880/orig.config.yaml 2021-03-08T07:40:58.875 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-03-07_01:01:03-crimson-rados-master-distro-basic-gibba/5942880 2021-03-08T07:40:58.876 INFO:teuthology.dispatcher.supervisor:Job PID: 8380 2021-03-08T07:40:58.876 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-08T19:41:48.959 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-08T19:41:49.214 INFO:teuthology.kill:Killing Pids: {8380} 2021-03-08T19:41:50.344 INFO:teuthology.kill:Nuking machines: ['gibba027', 'gibba036'] 2021-03-08T19:41:51.360 INFO:teuthology.kill:2021-03-08 19:41:51,359.359 INFO:teuthology.nuke:targets: 2021-03-08T19:41:51.361 INFO:teuthology.kill: gibba027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaVxVOI8PFj2njA1RQTFtoWvm68pJN3ovjeUyxkUroxk2wU/YxbocnGJfLYnSu3O0wgSjlfOvsWWQTqoqeMzx32h6be/f5IV9U583BL+UNO38laqUfxNKFhRWjSNXxOqPMw3pWMdD7DMfihoNWMfiPU/h/PdPhT05XE+A9yUEBjNxRlNH8v8ACtbC88ufWZLx6uvC4RvPqvyBC8OXLLvR6Umvb3A/MKRVoXCCxf18OsYI+lijoR9d+SMEAEK4sWBg+CfTuDgRBU3U2Yx8wSoNjBcZzRKsrjNPzQzHi1v8AJ82n99Y/lakHlbq7hZmenMM+iibLWro10tqiz564ouwq1YQ4l+wNLy28tMUQZWT5UN8d67aIsoum7q7Gs/R4gMfoon86pPS5X4s/lLoQzKiDAjy+W03r7O7nBcPYiy0C2T0V3WpAvVnXEhAVYDJQrtvd6sZqd7jA/Iv4K7uFI+6vr9WbSqinphZqDdC37YZoHma1M8uDLI3Yxn6VQQCqhtU= 2021-03-08T19:41:51.361 INFO:teuthology.kill: gibba036.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFqm42HMCD60IIET08K+qHXddSndak7FFkBR1uXsBHj5c2c46c+yR359wr5ULAmwv7Kx7P8MupAFnswtQe7lxRGRalZdsJGQDTBNN8ThFGJUGxInKIvwsCMVR2yuGG9BXONTCk40UEd1/HOeQUmU+MpHYd46WWrdiROFXGs5+weJfoC/f9evFNJfy2mBALr8WB7C9qtNeP2F34FjSrA7McJbNZ3Xbt9OeAdBe7amG0IVnQFs0+rmtwnirG7CqZxHBDBpq3YN7rPtYgDRwhUFtqsL3otOyR7vgs99tdKzwjWBrSpitJbEp7Rs192dpTDkIijxMKhVIq/2yrLo5s4/Zs3uWx8WvMIWvGX5T7+vpMX94KcijanyPGmcIsFOmm4ivfkE+AcJJs51PazYUWG8A4SBGnxKjKf1u6zgYd2i6QwpCX1qLmbt31zbx1nf/wBzkk0tokrCjD/SIb/VA06BjMJbY9MENKtCg3JAmSvOMoboyurdI0Lsn4g+BEF4KpS90= 2021-03-08T19:41:51.361 INFO:teuthology.kill:2021-03-08 19:41:51,361.361 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-08T19:41:51.367 INFO:teuthology.kill:2021-03-08 19:41:51,366.366 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-08T19:41:51.435 INFO:teuthology.kill:2021-03-08 19:41:51,435.435 INFO:teuthology.orchestra.console:Power off gibba036 2021-03-08T19:41:51.459 INFO:teuthology.kill:2021-03-08 19:41:51,458.458 INFO:teuthology.orchestra.console:Power off gibba027 2021-03-08T19:41:59.861 INFO:teuthology.kill:2021-03-08 19:41:59,860.860 INFO:teuthology.orchestra.console:Power off for gibba036 completed 2021-03-08T19:41:59.867 INFO:teuthology.kill:2021-03-08 19:41:59,867.867 INFO:teuthology.orchestra.console:Power off for gibba027 completed 2021-03-08T19:42:00.135 INFO:teuthology.task.internal:roles: ubuntu@gibba027.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2021-03-08T19:42:00.136 INFO:teuthology.task.internal:roles: ubuntu@gibba036.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2021-03-08T19:42:00.136 INFO:teuthology.misc:Compressing logs... 2021-03-08T19:42:00.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-08T19:42:34.691 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-08T19:42:34.836 INFO:teuthology.kill:No teuthology processes running 2021-03-08T19:42:36.590 INFO:teuthology.kill:Nuking machines: ['gibba027', 'gibba036'] 2021-03-08T19:42:37.893 INFO:teuthology.kill:2021-03-08 19:42:37,893.893 INFO:teuthology.nuke:targets: 2021-03-08T19:42:37.894 INFO:teuthology.kill: gibba027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaVxVOI8PFj2njA1RQTFtoWvm68pJN3ovjeUyxkUroxk2wU/YxbocnGJfLYnSu3O0wgSjlfOvsWWQTqoqeMzx32h6be/f5IV9U583BL+UNO38laqUfxNKFhRWjSNXxOqPMw3pWMdD7DMfihoNWMfiPU/h/PdPhT05XE+A9yUEBjNxRlNH8v8ACtbC88ufWZLx6uvC4RvPqvyBC8OXLLvR6Umvb3A/MKRVoXCCxf18OsYI+lijoR9d+SMEAEK4sWBg+CfTuDgRBU3U2Yx8wSoNjBcZzRKsrjNPzQzHi1v8AJ82n99Y/lakHlbq7hZmenMM+iibLWro10tqiz564ouwq1YQ4l+wNLy28tMUQZWT5UN8d67aIsoum7q7Gs/R4gMfoon86pPS5X4s/lLoQzKiDAjy+W03r7O7nBcPYiy0C2T0V3WpAvVnXEhAVYDJQrtvd6sZqd7jA/Iv4K7uFI+6vr9WbSqinphZqDdC37YZoHma1M8uDLI3Yxn6VQQCqhtU= 2021-03-08T19:42:37.894 INFO:teuthology.kill: gibba036.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFqm42HMCD60IIET08K+qHXddSndak7FFkBR1uXsBHj5c2c46c+yR359wr5ULAmwv7Kx7P8MupAFnswtQe7lxRGRalZdsJGQDTBNN8ThFGJUGxInKIvwsCMVR2yuGG9BXONTCk40UEd1/HOeQUmU+MpHYd46WWrdiROFXGs5+weJfoC/f9evFNJfy2mBALr8WB7C9qtNeP2F34FjSrA7McJbNZ3Xbt9OeAdBe7amG0IVnQFs0+rmtwnirG7CqZxHBDBpq3YN7rPtYgDRwhUFtqsL3otOyR7vgs99tdKzwjWBrSpitJbEp7Rs192dpTDkIijxMKhVIq/2yrLo5s4/Zs3uWx8WvMIWvGX5T7+vpMX94KcijanyPGmcIsFOmm4ivfkE+AcJJs51PazYUWG8A4SBGnxKjKf1u6zgYd2i6QwpCX1qLmbt31zbx1nf/wBzkk0tokrCjD/SIb/VA06BjMJbY9MENKtCg3JAmSvOMoboyurdI0Lsn4g+BEF4KpS90= 2021-03-08T19:42:37.915 INFO:teuthology.kill:2021-03-08 19:42:37,915.915 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-08T19:42:37.918 INFO:teuthology.kill:2021-03-08 19:42:37,918.918 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-08T19:42:38.012 INFO:teuthology.kill:2021-03-08 19:42:38,012.012 INFO:teuthology.orchestra.console:Power off gibba027 2021-03-08T19:42:38.078 INFO:teuthology.kill:2021-03-08 19:42:38,072.072 INFO:teuthology.orchestra.console:Power off gibba036 2021-03-08T19:42:38.279 INFO:teuthology.kill:2021-03-08 19:42:38,279.279 INFO:teuthology.orchestra.console:Power off for gibba027 completed 2021-03-08T19:42:38.285 INFO:teuthology.kill:2021-03-08 19:42:38,284.284 INFO:teuthology.orchestra.console:Power off for gibba036 completed 2021-03-08T19:42:38.425 INFO:teuthology.kill:2021-03-08 19:42:38,424.424 INFO:teuthology.lock.ops:unlocked gibba027.front.sepia.ceph.com 2021-03-08T19:42:38.428 INFO:teuthology.kill:2021-03-08 19:42:38,428.428 INFO:teuthology.lock.ops:unlocked gibba036.front.sepia.ceph.com 2021-03-08T19:44:38.875 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15