2021-07-04T14:14:33.486 INFO:root:teuthology version: 1.1.0-26631ede 2021-07-04T14:14:33.574 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba008.front.sepia.ceph.com' reimaging 2021-07-04T14:14:33.576 INFO:teuthology.lock.ops:Updating [ubuntu@gibba008.front.sepia.ceph.com]: reset os type and version on server 2021-07-04T14:14:33.576 INFO:teuthology.lock.ops:Updating gibba008.front.sepia.ceph.com on lock server 2021-07-04T14:14:33.592 INFO:teuthology.lock.ops:Node 'ubuntu@gibba008.front.sepia.ceph.com' reimaging is complete 2021-07-04T14:14:33.592 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba029.front.sepia.ceph.com' reimaging 2021-07-04T14:14:33.592 INFO:teuthology.lock.ops:Updating [ubuntu@gibba029.front.sepia.ceph.com]: reset os type and version on server 2021-07-04T14:14:33.593 INFO:teuthology.lock.ops:Updating gibba029.front.sepia.ceph.com on lock server 2021-07-04T14:14:33.607 INFO:teuthology.lock.ops:Node 'ubuntu@gibba029.front.sepia.ceph.com' reimaging is complete 2021-07-04T14:14:33.808 INFO:teuthology.provision.fog.gibba029:Scheduling deploy of centos 8.1 2021-07-04T14:14:33.813 INFO:teuthology.provision.fog.gibba008:Scheduling deploy of centos 8.1 2021-07-04T14:14:34.085 INFO:teuthology.orchestra.console:Power off gibba029 2021-07-04T14:14:34.117 INFO:teuthology.orchestra.console:Power off gibba008 2021-07-04T14:14:42.482 INFO:teuthology.orchestra.console:Power off for gibba008 completed 2021-07-04T14:14:42.583 INFO:teuthology.orchestra.console:Power on gibba008 2021-07-04T14:14:46.588 INFO:teuthology.orchestra.console:Power off for gibba029 completed 2021-07-04T14:14:46.689 INFO:teuthology.orchestra.console:Power on gibba029 2021-07-04T14:14:51.022 INFO:teuthology.orchestra.console:Power on for gibba008 completed 2021-07-04T14:14:51.123 INFO:teuthology.provision.fog.gibba008:Waiting for deploy to finish 2021-07-04T14:14:55.065 INFO:teuthology.orchestra.console:Power on for gibba029 completed 2021-07-04T14:14:55.166 INFO:teuthology.provision.fog.gibba029:Waiting for deploy to finish 2021-07-04T14:18:48.397 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-04T14:18:58.615 INFO:teuthology.provision.fog.gibba008:Node is ready 2021-07-04T14:18:58.629 INFO:teuthology.orchestra.run.gibba008.stdout:gibba029 2021-07-04T14:18:58.682 INFO:teuthology.orchestra.run.gibba008.stdout:172.21.2.129 gibba029.front.sepia.ceph.com gibba029 2021-07-04T14:18:59.171 INFO:teuthology.provision.fog.gibba008:Deploy complete! 2021-07-04T14:19:26.860 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-04T14:19:27.146 INFO:teuthology.provision.fog.gibba029:Node is ready 2021-07-04T14:19:27.157 INFO:teuthology.orchestra.run.gibba029.stdout:gibba029.front.sepia.ceph.com 2021-07-04T14:19:27.210 INFO:teuthology.orchestra.run.gibba029.stdout:172.21.2.129 gibba029.front.sepia.ceph.com gibba029 2021-07-04T14:19:27.520 INFO:teuthology.provision.fog.gibba029:Deploy complete! 2021-07-04T14:19:27.643 INFO:teuthology.lock.ops:Checking gibba008.front.sepia.ceph.com 2021-07-04T14:19:27.644 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-04T14:19:27.659 INFO:teuthology.lock.ops:Checking gibba029.front.sepia.ceph.com 2021-07-04T14:19:27.659 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-04T14:19:27.675 INFO:teuthology.lock.ops:Updating [gibba008.front.sepia.ceph.com]: set os type and version on server 2021-07-04T14:19:27.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-04T14:19:27.924 INFO:teuthology.orchestra.run.gibba008.stdout:x86_64 2021-07-04T14:19:27.977 INFO:teuthology.orchestra.run.gibba008.stdout:NAME="CentOS Linux" 2021-07-04T14:19:27.978 INFO:teuthology.orchestra.run.gibba008.stdout:VERSION="8 (Core)" 2021-07-04T14:19:27.978 INFO:teuthology.orchestra.run.gibba008.stdout:ID="centos" 2021-07-04T14:19:27.978 INFO:teuthology.orchestra.run.gibba008.stdout:ID_LIKE="rhel fedora" 2021-07-04T14:19:27.979 INFO:teuthology.orchestra.run.gibba008.stdout:VERSION_ID="8" 2021-07-04T14:19:27.979 INFO:teuthology.orchestra.run.gibba008.stdout:PLATFORM_ID="platform:el8" 2021-07-04T14:19:27.979 INFO:teuthology.orchestra.run.gibba008.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-07-04T14:19:27.979 INFO:teuthology.orchestra.run.gibba008.stdout:ANSI_COLOR="0;31" 2021-07-04T14:19:27.979 INFO:teuthology.orchestra.run.gibba008.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-07-04T14:19:27.980 INFO:teuthology.orchestra.run.gibba008.stdout:HOME_URL="https://www.centos.org/" 2021-07-04T14:19:27.980 INFO:teuthology.orchestra.run.gibba008.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-07-04T14:19:27.980 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-07-04T14:19:27.980 INFO:teuthology.orchestra.run.gibba008.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-07-04T14:19:27.981 INFO:teuthology.orchestra.run.gibba008.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-07-04T14:19:27.981 INFO:teuthology.orchestra.run.gibba008.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-07-04T14:19:27.981 INFO:teuthology.orchestra.run.gibba008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-07-04T14:19:27.981 INFO:teuthology.orchestra.run.gibba008.stdout: 2021-07-04T14:19:27.982 INFO:teuthology.lock.ops:Updating gibba008.front.sepia.ceph.com on lock server 2021-07-04T14:19:27.997 INFO:teuthology.lock.ops:Updating [gibba029.front.sepia.ceph.com]: set os type and version on server 2021-07-04T14:19:27.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-04T14:19:28.243 INFO:teuthology.orchestra.run.gibba029.stdout:x86_64 2021-07-04T14:19:28.295 INFO:teuthology.orchestra.run.gibba029.stdout:NAME="CentOS Linux" 2021-07-04T14:19:28.296 INFO:teuthology.orchestra.run.gibba029.stdout:VERSION="8 (Core)" 2021-07-04T14:19:28.296 INFO:teuthology.orchestra.run.gibba029.stdout:ID="centos" 2021-07-04T14:19:28.296 INFO:teuthology.orchestra.run.gibba029.stdout:ID_LIKE="rhel fedora" 2021-07-04T14:19:28.296 INFO:teuthology.orchestra.run.gibba029.stdout:VERSION_ID="8" 2021-07-04T14:19:28.296 INFO:teuthology.orchestra.run.gibba029.stdout:PLATFORM_ID="platform:el8" 2021-07-04T14:19:28.297 INFO:teuthology.orchestra.run.gibba029.stdout:PRETTY_NAME="CentOS Linux 8 (Core)" 2021-07-04T14:19:28.297 INFO:teuthology.orchestra.run.gibba029.stdout:ANSI_COLOR="0;31" 2021-07-04T14:19:28.297 INFO:teuthology.orchestra.run.gibba029.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-07-04T14:19:28.297 INFO:teuthology.orchestra.run.gibba029.stdout:HOME_URL="https://www.centos.org/" 2021-07-04T14:19:28.297 INFO:teuthology.orchestra.run.gibba029.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-07-04T14:19:28.298 INFO:teuthology.orchestra.run.gibba029.stdout: 2021-07-04T14:19:28.298 INFO:teuthology.orchestra.run.gibba029.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-07-04T14:19:28.298 INFO:teuthology.orchestra.run.gibba029.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-07-04T14:19:28.298 INFO:teuthology.orchestra.run.gibba029.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2021-07-04T14:19:28.299 INFO:teuthology.orchestra.run.gibba029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8" 2021-07-04T14:19:28.299 INFO:teuthology.orchestra.run.gibba029.stdout: 2021-07-04T14:19:28.299 INFO:teuthology.lock.ops:Updating gibba029.front.sepia.ceph.com on lock server 2021-07-04T14:19:28.375 INFO:teuthology.dispatcher.supervisor:Running job 6188446 2021-07-04T14:19:28.375 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-06-24_06:00:04-rbd-octopus-distro-basic-gibba/6188446 --name teuthology-2021-06-24_06:00:04-rbd-octopus-distro-basic-gibba --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-hybrid policy/none rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum} -- /home/teuthworker/archive/teuthology-2021-06-24_06:00:04-rbd-octopus-distro-basic-gibba/6188446/orig.config.yaml 2021-07-04T14:19:28.379 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-06-24_06:00:04-rbd-octopus-distro-basic-gibba/6188446 2021-07-04T14:19:28.380 INFO:teuthology.dispatcher.supervisor:Job PID: 1304 2021-07-04T14:19:28.380 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-07-05T02:20:10.613 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-07-05T02:20:10.812 INFO:teuthology.kill:Killing Pids: {1304} 2021-07-05T02:20:11.784 INFO:teuthology.kill:Nuking machines: ['gibba008', 'gibba029'] 2021-07-05T02:20:12.607 INFO:teuthology.kill:2021-07-05 02:20:12,607.607 INFO:teuthology.nuke:targets: 2021-07-05T02:20:12.608 INFO:teuthology.kill: gibba008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSV3ow6wgdrpVunrLTspdDGQyjOubMkO9SIUlwCJjQmA5WBbHODXPRGbOQPgiJqOmy8llxJrgbnV7pg9agM+2D7WtnN3VBbGVXoVUYIAUce3IsuQYdoY0hd5t+KReimH3FN2uy3ZKa+a5ChIGhdlMb0aQ6igd60DU8RTsZX1bJazIybLQ3p5TJ/50l728xj/Ew81ZcXx25p2248HQHsQWtrxa22Vh/HbZbq+QvQi1FgqhHfMdtuoCtSn5CwGGeAy/qyuMTgOFqs0oArfj0b1K8WS1uPDsUdDiza72V65NFX1D/b0xhiTU/N3905fkjCPf9EDkgLWfnKjHCiQfvrp60OafMbQYykL+WCQ7EM97bRBdQExRDVdLxDkKVe2xEtB5sYK/sAsJ8slFIL87wuJ8yIQPljs8TSIqCv4DvFtT8BF3RSDujj4qDXNQ4VyMN9/7fJbX2UYf8xoGtOhfhSovqBmxGwxAI60ahrFXYoUJ857T21sDJPJiKtzSvQIM+zv8= 2021-07-05T02:20:12.608 INFO:teuthology.kill: gibba029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDK3hsk3d1aMW0qxZHmGunKGBFlX8MZqgnP2FkbqZ6sFFunzdRx3Z2++SHfGU1uP/hZC1mWMGq1Rd+GvMZKP7TpzvmVluimy5OzqOjxKcS1GDrCIAfOmZcnNMBztcBwiXu1SqqJjkU/Pphsbd5hTsydoxMnFCTOt83dSp/+Ylh/Ops+Lx2irQyYqEyGTYRTrOnsR/Om5EfHCeqyUWeAGdCvYgjNoGTs0QcLGBYKIMuFs8bcLTUlhxWENLLbi9IIVuBoX15PhzqfKA8dd2pdACcQ7kxKy/gD2jR3IbrUG+hS4CkVk7TrimFy9SfuoWSRH71sPhXIxD2wBKBoIy71Tuta/1w4hBG5aOuDJ0d6fvvC7UGQIsFuLbV13/rdzyGN+4ysa0enEYqVPTkYX5Jjnrt5bzuaQkfM/n/k5Hp5gXavwNZC7H35TfoAlXY/dPY4yrJKIm91dMS9+jBDjm1e95B7DVP5JaR123Ykpq94ob0t5tVPak/QjG9s0e3zpkedhDM= 2021-07-05T02:20:12.608 INFO:teuthology.kill:2021-07-05 02:20:12,607.607 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-05T02:20:12.609 INFO:teuthology.kill:2021-07-05 02:20:12,609.609 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-05T02:20:12.664 INFO:teuthology.kill:2021-07-05 02:20:12,664.664 INFO:teuthology.orchestra.console:Power off gibba008 2021-07-05T02:20:12.677 INFO:teuthology.kill:2021-07-05 02:20:12,677.677 INFO:teuthology.orchestra.console:Power off gibba029 2021-07-05T02:20:21.044 INFO:teuthology.kill:2021-07-05 02:20:21,044.044 INFO:teuthology.orchestra.console:Power off for gibba008 completed 2021-07-05T02:20:25.239 INFO:teuthology.kill:2021-07-05 02:20:25,239.239 INFO:teuthology.orchestra.console:Power off for gibba029 completed 2021-07-05T02:20:25.444 INFO:teuthology.task.internal:roles: ubuntu@gibba008.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2021-07-05T02:20:25.444 INFO:teuthology.task.internal:roles: ubuntu@gibba029.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2021-07-05T02:20:25.445 INFO:teuthology.misc:Compressing logs... 2021-07-05T02:20:25.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-05T02:21:25.503 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-07-05T02:21:25.564 INFO:teuthology.kill:No teuthology processes running 2021-07-05T02:21:26.543 INFO:teuthology.kill:Nuking machines: ['gibba008', 'gibba029'] 2021-07-05T02:21:27.379 INFO:teuthology.kill:2021-07-05 02:21:27,378.378 INFO:teuthology.nuke:targets: 2021-07-05T02:21:27.379 INFO:teuthology.kill: gibba008.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSV3ow6wgdrpVunrLTspdDGQyjOubMkO9SIUlwCJjQmA5WBbHODXPRGbOQPgiJqOmy8llxJrgbnV7pg9agM+2D7WtnN3VBbGVXoVUYIAUce3IsuQYdoY0hd5t+KReimH3FN2uy3ZKa+a5ChIGhdlMb0aQ6igd60DU8RTsZX1bJazIybLQ3p5TJ/50l728xj/Ew81ZcXx25p2248HQHsQWtrxa22Vh/HbZbq+QvQi1FgqhHfMdtuoCtSn5CwGGeAy/qyuMTgOFqs0oArfj0b1K8WS1uPDsUdDiza72V65NFX1D/b0xhiTU/N3905fkjCPf9EDkgLWfnKjHCiQfvrp60OafMbQYykL+WCQ7EM97bRBdQExRDVdLxDkKVe2xEtB5sYK/sAsJ8slFIL87wuJ8yIQPljs8TSIqCv4DvFtT8BF3RSDujj4qDXNQ4VyMN9/7fJbX2UYf8xoGtOhfhSovqBmxGwxAI60ahrFXYoUJ857T21sDJPJiKtzSvQIM+zv8= 2021-07-05T02:21:27.380 INFO:teuthology.kill: gibba029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDK3hsk3d1aMW0qxZHmGunKGBFlX8MZqgnP2FkbqZ6sFFunzdRx3Z2++SHfGU1uP/hZC1mWMGq1Rd+GvMZKP7TpzvmVluimy5OzqOjxKcS1GDrCIAfOmZcnNMBztcBwiXu1SqqJjkU/Pphsbd5hTsydoxMnFCTOt83dSp/+Ylh/Ops+Lx2irQyYqEyGTYRTrOnsR/Om5EfHCeqyUWeAGdCvYgjNoGTs0QcLGBYKIMuFs8bcLTUlhxWENLLbi9IIVuBoX15PhzqfKA8dd2pdACcQ7kxKy/gD2jR3IbrUG+hS4CkVk7TrimFy9SfuoWSRH71sPhXIxD2wBKBoIy71Tuta/1w4hBG5aOuDJ0d6fvvC7UGQIsFuLbV13/rdzyGN+4ysa0enEYqVPTkYX5Jjnrt5bzuaQkfM/n/k5Hp5gXavwNZC7H35TfoAlXY/dPY4yrJKIm91dMS9+jBDjm1e95B7DVP5JaR123Ykpq94ob0t5tVPak/QjG9s0e3zpkedhDM= 2021-07-05T02:21:27.398 INFO:teuthology.kill:2021-07-05 02:21:27,398.398 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-05T02:21:27.401 INFO:teuthology.kill:2021-07-05 02:21:27,401.401 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-05T02:21:27.454 INFO:teuthology.kill:2021-07-05 02:21:27,454.454 INFO:teuthology.orchestra.console:Power off gibba029 2021-07-05T02:21:27.467 INFO:teuthology.kill:2021-07-05 02:21:27,467.467 INFO:teuthology.orchestra.console:Power off gibba008 2021-07-05T02:21:27.604 INFO:teuthology.kill:2021-07-05 02:21:27,604.604 INFO:teuthology.orchestra.console:Power off for gibba029 completed 2021-07-05T02:21:27.615 INFO:teuthology.kill:2021-07-05 02:21:27,614.614 INFO:teuthology.orchestra.console:Power off for gibba008 completed 2021-07-05T02:21:27.747 INFO:teuthology.kill:2021-07-05 02:21:27,746.746 INFO:teuthology.lock.ops:unlocked gibba029.front.sepia.ceph.com 2021-07-05T02:21:27.761 INFO:teuthology.kill:2021-07-05 02:21:27,761.761 INFO:teuthology.lock.ops:unlocked gibba008.front.sepia.ceph.com 2021-07-05T02:23:28.020 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15