2021-07-20T15:32:46.606 INFO:root:teuthology version: 1.1.0-43790980 2021-07-20T15:32:46.760 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba015.front.sepia.ceph.com' reimaging 2021-07-20T15:32:46.761 INFO:teuthology.lock.ops:Updating [ubuntu@gibba015.front.sepia.ceph.com]: reset os type and version on server 2021-07-20T15:32:46.761 INFO:teuthology.lock.ops:Updating gibba015.front.sepia.ceph.com on lock server 2021-07-20T15:32:46.776 INFO:teuthology.lock.ops:Node 'ubuntu@gibba015.front.sepia.ceph.com' reimaging is complete 2021-07-20T15:32:46.776 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba028.front.sepia.ceph.com' reimaging 2021-07-20T15:32:46.776 INFO:teuthology.lock.ops:Updating [ubuntu@gibba028.front.sepia.ceph.com]: reset os type and version on server 2021-07-20T15:32:46.777 INFO:teuthology.lock.ops:Updating gibba028.front.sepia.ceph.com on lock server 2021-07-20T15:32:46.791 INFO:teuthology.lock.ops:Node 'ubuntu@gibba028.front.sepia.ceph.com' reimaging is complete 2021-07-20T15:32:46.791 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba029.front.sepia.ceph.com' reimaging 2021-07-20T15:32:46.791 INFO:teuthology.lock.ops:Updating [ubuntu@gibba029.front.sepia.ceph.com]: reset os type and version on server 2021-07-20T15:32:46.792 INFO:teuthology.lock.ops:Updating gibba029.front.sepia.ceph.com on lock server 2021-07-20T15:32:46.805 INFO:teuthology.lock.ops:Node 'ubuntu@gibba029.front.sepia.ceph.com' reimaging is complete 2021-07-20T15:32:46.805 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba035.front.sepia.ceph.com' reimaging 2021-07-20T15:32:46.806 INFO:teuthology.lock.ops:Updating [ubuntu@gibba035.front.sepia.ceph.com]: reset os type and version on server 2021-07-20T15:32:46.806 INFO:teuthology.lock.ops:Updating gibba035.front.sepia.ceph.com on lock server 2021-07-20T15:32:46.821 INFO:teuthology.lock.ops:Node 'ubuntu@gibba035.front.sepia.ceph.com' reimaging is complete 2021-07-20T15:32:46.821 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba040.front.sepia.ceph.com' reimaging 2021-07-20T15:32:46.822 INFO:teuthology.lock.ops:Updating [ubuntu@gibba040.front.sepia.ceph.com]: reset os type and version on server 2021-07-20T15:32:46.822 INFO:teuthology.lock.ops:Updating gibba040.front.sepia.ceph.com on lock server 2021-07-20T15:32:46.835 INFO:teuthology.lock.ops:Node 'ubuntu@gibba040.front.sepia.ceph.com' reimaging is complete 2021-07-20T15:32:46.999 INFO:teuthology.provision.fog.gibba028:Scheduling deploy of ubuntu 20.04 2021-07-20T15:32:47.012 INFO:teuthology.provision.fog.gibba015:Scheduling deploy of ubuntu 20.04 2021-07-20T15:32:47.014 INFO:teuthology.provision.fog.gibba029:Scheduling deploy of ubuntu 20.04 2021-07-20T15:32:47.030 INFO:teuthology.provision.fog.gibba035:Scheduling deploy of ubuntu 20.04 2021-07-20T15:32:47.050 INFO:teuthology.provision.fog.gibba040:Scheduling deploy of ubuntu 20.04 2021-07-20T15:32:47.302 INFO:teuthology.orchestra.console:Power off gibba028 2021-07-20T15:32:47.313 INFO:teuthology.orchestra.console:Power off gibba015 2021-07-20T15:32:47.367 INFO:teuthology.orchestra.console:Power off gibba029 2021-07-20T15:32:47.380 INFO:teuthology.orchestra.console:Power off gibba035 2021-07-20T15:32:47.421 INFO:teuthology.orchestra.console:Power off gibba040 2021-07-20T15:32:55.688 INFO:teuthology.orchestra.console:Power off for gibba028 completed 2021-07-20T15:32:55.705 INFO:teuthology.orchestra.console:Power off for gibba015 completed 2021-07-20T15:32:55.733 INFO:teuthology.orchestra.console:Power off for gibba029 completed 2021-07-20T15:32:55.745 INFO:teuthology.orchestra.console:Power off for gibba035 completed 2021-07-20T15:32:55.786 INFO:teuthology.orchestra.console:Power off for gibba040 completed 2021-07-20T15:32:55.789 INFO:teuthology.orchestra.console:Power on gibba028 2021-07-20T15:32:55.806 INFO:teuthology.orchestra.console:Power on gibba015 2021-07-20T15:32:55.836 INFO:teuthology.orchestra.console:Power on gibba029 2021-07-20T15:32:55.847 INFO:teuthology.orchestra.console:Power on gibba035 2021-07-20T15:32:55.886 INFO:teuthology.orchestra.console:Power on gibba040 2021-07-20T15:33:04.169 INFO:teuthology.orchestra.console:Power on for gibba028 completed 2021-07-20T15:33:04.173 INFO:teuthology.orchestra.console:Power on for gibba015 completed 2021-07-20T15:33:04.198 INFO:teuthology.orchestra.console:Power on for gibba029 completed 2021-07-20T15:33:04.210 INFO:teuthology.orchestra.console:Power on for gibba035 completed 2021-07-20T15:33:04.245 INFO:teuthology.orchestra.console:Power on for gibba040 completed 2021-07-20T15:33:04.270 INFO:teuthology.provision.fog.gibba028:Waiting for deploy to finish 2021-07-20T15:33:04.274 INFO:teuthology.provision.fog.gibba015:Waiting for deploy to finish 2021-07-20T15:33:04.299 INFO:teuthology.provision.fog.gibba029:Waiting for deploy to finish 2021-07-20T15:33:04.312 INFO:teuthology.provision.fog.gibba035:Waiting for deploy to finish 2021-07-20T15:33:04.346 INFO:teuthology.provision.fog.gibba040:Waiting for deploy to finish 2021-07-20T15:37:44.522 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-20T15:37:44.525 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-20T15:37:44.528 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-20T15:37:44.828 INFO:teuthology.provision.fog.gibba040:Node is ready 2021-07-20T15:37:44.834 INFO:teuthology.orchestra.run.gibba040.stdout:gibba040.front.sepia.ceph.com 2021-07-20T15:37:44.842 INFO:teuthology.orchestra.run.gibba040.stdout:172.21.2.140 gibba040.front.sepia.ceph.com gibba040 2021-07-20T15:37:44.927 INFO:teuthology.provision.fog.gibba029:Node is ready 2021-07-20T15:37:44.930 INFO:teuthology.orchestra.run.gibba029.stdout:gibba029.front.sepia.ceph.com 2021-07-20T15:37:44.934 INFO:teuthology.orchestra.run.gibba029.stdout:172.21.2.129 gibba029.front.sepia.ceph.com gibba029 2021-07-20T15:37:44.971 INFO:teuthology.provision.fog.gibba040:Deploy complete! 2021-07-20T15:37:45.129 INFO:teuthology.provision.fog.gibba029:Deploy complete! 2021-07-20T15:37:45.320 INFO:teuthology.provision.fog.gibba035:Node is ready 2021-07-20T15:37:45.329 INFO:teuthology.orchestra.run.gibba035.stdout:gibba035.front.sepia.ceph.com 2021-07-20T15:37:45.338 INFO:teuthology.orchestra.run.gibba035.stdout:172.21.2.135 gibba035.front.sepia.ceph.com gibba035 2021-07-20T15:37:45.449 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-20T15:37:45.504 INFO:teuthology.provision.fog.gibba035:Deploy complete! 2021-07-20T15:37:45.792 INFO:teuthology.provision.fog.gibba028:Node is ready 2021-07-20T15:37:45.797 INFO:teuthology.orchestra.run.gibba028.stdout:gibba028.front.sepia.ceph.com 2021-07-20T15:37:45.846 INFO:teuthology.orchestra.run.gibba028.stdout:172.21.2.128 gibba028.front.sepia.ceph.com gibba028 2021-07-20T15:37:45.960 INFO:teuthology.provision.fog.gibba028:Deploy complete! 2021-07-20T15:37:49.490 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-07-20T15:37:49.878 INFO:teuthology.provision.fog.gibba015:Node is ready 2021-07-20T15:37:49.882 INFO:teuthology.orchestra.run.gibba015.stdout:gibba015.front.sepia.ceph.com 2021-07-20T15:37:49.886 INFO:teuthology.orchestra.run.gibba015.stdout:172.21.2.115 gibba015.front.sepia.ceph.com gibba015 2021-07-20T15:37:50.081 INFO:teuthology.provision.fog.gibba015:Deploy complete! 2021-07-20T15:37:50.305 INFO:teuthology.lock.ops:Checking gibba015.front.sepia.ceph.com 2021-07-20T15:37:50.306 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-20T15:37:50.322 INFO:teuthology.lock.ops:Checking gibba028.front.sepia.ceph.com 2021-07-20T15:37:50.322 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-20T15:37:50.334 INFO:teuthology.lock.ops:Checking gibba029.front.sepia.ceph.com 2021-07-20T15:37:50.334 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-20T15:37:50.349 INFO:teuthology.lock.ops:Checking gibba035.front.sepia.ceph.com 2021-07-20T15:37:50.350 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-20T15:37:50.364 INFO:teuthology.lock.ops:Checking gibba040.front.sepia.ceph.com 2021-07-20T15:37:50.364 INFO:teuthology.lock.ops:New key found. Updating... 2021-07-20T15:37:50.376 INFO:teuthology.lock.ops:Updating [gibba015.front.sepia.ceph.com]: set os type and version on server 2021-07-20T15:37:50.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-20T15:37:50.598 INFO:teuthology.orchestra.run.gibba015.stdout:x86_64 2021-07-20T15:37:50.602 INFO:teuthology.orchestra.run.gibba015.stdout:NAME="Ubuntu" 2021-07-20T15:37:50.603 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-20T15:37:50.603 INFO:teuthology.orchestra.run.gibba015.stdout:ID=ubuntu 2021-07-20T15:37:50.603 INFO:teuthology.orchestra.run.gibba015.stdout:ID_LIKE=debian 2021-07-20T15:37:50.603 INFO:teuthology.orchestra.run.gibba015.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-20T15:37:50.604 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION_ID="20.04" 2021-07-20T15:37:50.604 INFO:teuthology.orchestra.run.gibba015.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-20T15:37:50.604 INFO:teuthology.orchestra.run.gibba015.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-20T15:37:50.604 INFO:teuthology.orchestra.run.gibba015.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-20T15:37:50.604 INFO:teuthology.orchestra.run.gibba015.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-20T15:37:50.605 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION_CODENAME=focal 2021-07-20T15:37:50.605 INFO:teuthology.orchestra.run.gibba015.stdout:UBUNTU_CODENAME=focal 2021-07-20T15:37:50.606 INFO:teuthology.lock.ops:Updating gibba015.front.sepia.ceph.com on lock server 2021-07-20T15:37:50.623 INFO:teuthology.lock.ops:Updating [gibba028.front.sepia.ceph.com]: set os type and version on server 2021-07-20T15:37:50.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-20T15:37:50.737 INFO:teuthology.orchestra.run.gibba028.stdout:x86_64 2021-07-20T15:37:50.742 INFO:teuthology.orchestra.run.gibba028.stdout:NAME="Ubuntu" 2021-07-20T15:37:50.742 INFO:teuthology.orchestra.run.gibba028.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-20T15:37:50.742 INFO:teuthology.orchestra.run.gibba028.stdout:ID=ubuntu 2021-07-20T15:37:50.743 INFO:teuthology.orchestra.run.gibba028.stdout:ID_LIKE=debian 2021-07-20T15:37:50.743 INFO:teuthology.orchestra.run.gibba028.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-20T15:37:50.743 INFO:teuthology.orchestra.run.gibba028.stdout:VERSION_ID="20.04" 2021-07-20T15:37:50.743 INFO:teuthology.orchestra.run.gibba028.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-20T15:37:50.743 INFO:teuthology.orchestra.run.gibba028.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-20T15:37:50.744 INFO:teuthology.orchestra.run.gibba028.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-20T15:37:50.744 INFO:teuthology.orchestra.run.gibba028.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-20T15:37:50.744 INFO:teuthology.orchestra.run.gibba028.stdout:VERSION_CODENAME=focal 2021-07-20T15:37:50.744 INFO:teuthology.orchestra.run.gibba028.stdout:UBUNTU_CODENAME=focal 2021-07-20T15:37:50.745 INFO:teuthology.lock.ops:Updating gibba028.front.sepia.ceph.com on lock server 2021-07-20T15:37:50.758 INFO:teuthology.lock.ops:Updating [gibba029.front.sepia.ceph.com]: set os type and version on server 2021-07-20T15:37:50.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-20T15:37:50.877 INFO:teuthology.orchestra.run.gibba029.stdout:x86_64 2021-07-20T15:37:50.881 INFO:teuthology.orchestra.run.gibba029.stdout:NAME="Ubuntu" 2021-07-20T15:37:50.882 INFO:teuthology.orchestra.run.gibba029.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-20T15:37:50.882 INFO:teuthology.orchestra.run.gibba029.stdout:ID=ubuntu 2021-07-20T15:37:50.882 INFO:teuthology.orchestra.run.gibba029.stdout:ID_LIKE=debian 2021-07-20T15:37:50.883 INFO:teuthology.orchestra.run.gibba029.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-20T15:37:50.883 INFO:teuthology.orchestra.run.gibba029.stdout:VERSION_ID="20.04" 2021-07-20T15:37:50.883 INFO:teuthology.orchestra.run.gibba029.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-20T15:37:50.884 INFO:teuthology.orchestra.run.gibba029.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-20T15:37:50.884 INFO:teuthology.orchestra.run.gibba029.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-20T15:37:50.884 INFO:teuthology.orchestra.run.gibba029.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-20T15:37:50.884 INFO:teuthology.orchestra.run.gibba029.stdout:VERSION_CODENAME=focal 2021-07-20T15:37:50.885 INFO:teuthology.orchestra.run.gibba029.stdout:UBUNTU_CODENAME=focal 2021-07-20T15:37:50.885 INFO:teuthology.lock.ops:Updating gibba029.front.sepia.ceph.com on lock server 2021-07-20T15:37:50.900 INFO:teuthology.lock.ops:Updating [gibba035.front.sepia.ceph.com]: set os type and version on server 2021-07-20T15:37:50.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-20T15:37:51.061 INFO:teuthology.orchestra.run.gibba035.stdout:x86_64 2021-07-20T15:37:51.067 INFO:teuthology.orchestra.run.gibba035.stdout:NAME="Ubuntu" 2021-07-20T15:37:51.068 INFO:teuthology.orchestra.run.gibba035.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-20T15:37:51.068 INFO:teuthology.orchestra.run.gibba035.stdout:ID=ubuntu 2021-07-20T15:37:51.068 INFO:teuthology.orchestra.run.gibba035.stdout:ID_LIKE=debian 2021-07-20T15:37:51.068 INFO:teuthology.orchestra.run.gibba035.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-20T15:37:51.068 INFO:teuthology.orchestra.run.gibba035.stdout:VERSION_ID="20.04" 2021-07-20T15:37:51.069 INFO:teuthology.orchestra.run.gibba035.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-20T15:37:51.069 INFO:teuthology.orchestra.run.gibba035.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-20T15:37:51.069 INFO:teuthology.orchestra.run.gibba035.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-20T15:37:51.069 INFO:teuthology.orchestra.run.gibba035.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-20T15:37:51.069 INFO:teuthology.orchestra.run.gibba035.stdout:VERSION_CODENAME=focal 2021-07-20T15:37:51.070 INFO:teuthology.orchestra.run.gibba035.stdout:UBUNTU_CODENAME=focal 2021-07-20T15:37:51.070 INFO:teuthology.lock.ops:Updating gibba035.front.sepia.ceph.com on lock server 2021-07-20T15:37:51.085 INFO:teuthology.lock.ops:Updating [gibba040.front.sepia.ceph.com]: set os type and version on server 2021-07-20T15:37:51.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-20T15:37:51.201 INFO:teuthology.orchestra.run.gibba040.stdout:x86_64 2021-07-20T15:37:51.206 INFO:teuthology.orchestra.run.gibba040.stdout:NAME="Ubuntu" 2021-07-20T15:37:51.206 INFO:teuthology.orchestra.run.gibba040.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-20T15:37:51.206 INFO:teuthology.orchestra.run.gibba040.stdout:ID=ubuntu 2021-07-20T15:37:51.207 INFO:teuthology.orchestra.run.gibba040.stdout:ID_LIKE=debian 2021-07-20T15:37:51.207 INFO:teuthology.orchestra.run.gibba040.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-20T15:37:51.207 INFO:teuthology.orchestra.run.gibba040.stdout:VERSION_ID="20.04" 2021-07-20T15:37:51.207 INFO:teuthology.orchestra.run.gibba040.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-20T15:37:51.208 INFO:teuthology.orchestra.run.gibba040.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-20T15:37:51.208 INFO:teuthology.orchestra.run.gibba040.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-20T15:37:51.208 INFO:teuthology.orchestra.run.gibba040.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-20T15:37:51.208 INFO:teuthology.orchestra.run.gibba040.stdout:VERSION_CODENAME=focal 2021-07-20T15:37:51.208 INFO:teuthology.orchestra.run.gibba040.stdout:UBUNTU_CODENAME=focal 2021-07-20T15:37:51.209 INFO:teuthology.lock.ops:Updating gibba040.front.sepia.ceph.com on lock server 2021-07-20T15:37:51.292 INFO:teuthology.dispatcher.supervisor:Running job 6275369 2021-07-20T15:37:51.292 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_43790980b608e9a71ad69685c3f7e95578734f1b/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-07-16_04:10:03-fs-octopus-distro-basic-gibba/6275369 --name teuthology-2021-07-16_04:10:03-fs-octopus-distro-basic-gibba --description fs/multiclient/{begin clusters/1-mds-3-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-ec-root overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} -- /home/teuthworker/archive/teuthology-2021-07-16_04:10:03-fs-octopus-distro-basic-gibba/6275369/orig.config.yaml 2021-07-20T15:37:51.296 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-07-16_04:10:03-fs-octopus-distro-basic-gibba/6275369 2021-07-20T15:37:51.297 INFO:teuthology.dispatcher.supervisor:Job PID: 14754 2021-07-20T15:37:51.297 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-07-21T03:38:39.924 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-07-21T03:38:40.205 INFO:teuthology.kill:Killing Pids: {14754} 2021-07-21T03:38:41.898 INFO:teuthology.kill:Nuking machines: ['gibba015', 'gibba028', 'gibba029', 'gibba035', 'gibba040'] 2021-07-21T03:38:42.708 INFO:teuthology.kill:2021-07-21 03:38:42,708.708 INFO:teuthology.nuke:targets: 2021-07-21T03:38:42.709 INFO:teuthology.kill: gibba015.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHWL2R0ErOtc2mvnunbyhx1L5+W3q+Q+iYyI1Plzhv8u9yGXUakA/D6R3bmGI7SZdHEyzLjFVLOtdz35P+EGFgtHQtOPSubQRu2ZwiMibvzLaj2e7TXQ/6cvhJ0ouse0QXng5fMNrP6s28bqWy8G3BPeWkprVOk6OElu4nuHw2d8KxQjkp5uY4CunZhXahSBH01M9Yv5rXyS2MDqQtSzaZ3ptGxZDhqDd5LwMfQ1YU2QARbzB/oBV/2XHMVdiFmczQvVpji3cT0piOUNOmb+cSG86epMLdJJ0+U4DScNMi9BEoUcACw/yEbYRP6j5aBemhMGe/UdlMLh5AWQpOSyugDXLgJ+PDDPIiCSENSekrLIkv9ZVy1qCYyArwhL8bCDoTeDAQKjWfMv/gV4khv7tNaqFzorWHKbbE4aMBDVkID7oinA800pYfzXHlXKPh9O80up/KECO9+RkecuM6Si0oKWblrgWK/Dxe3ZorWlLRyL4HQx7jt23aWyIjHPLWnVM= 2021-07-21T03:38:42.710 INFO:teuthology.kill: gibba028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzaQs7cSZZFYjlWlpiq6Jbc406g0FbW3nmiB4WzyUuxedn8DJNINZtdeWdTIDKzbM/I+sSQdP8aXdTu+yrvgcdEeCIBE6l6BTlBIytzWwoHzDkGta9MJtMdkk3fHxy6P8YOzM+qFPAYaJ5kMWdH74rZxqQqw+4jvvLJW3RCe+6Bn2LiFKQnIfD/5nAJLWSC+ei/gWnwFiM6zYph1RODpxHgAoA7zj7dP7EnEa6gF+cD4bqbLPDSJjJIicBHFnVzz3inpdd9qQHiamJViFpMKS4hLGwh7uR6voRXbTSeMSHis8eTHXlA9lnd5vtTKdaNimR0fHs9Fa/tCMeoKV7QWyApUk1HItui0g1Ud84rvm7NvK0GPJoD2AzLNXCyWk1GMZ1Ay4GFEM7MkRcDV0KPrQWKUhAfQvZZYmHeV0i3oLCtEIcSnmoGmvu1GPyiGB55guXJwF5IEf5FOxuCriB+icrfT5CkwdeWTH7SCs7fBvs8tQvLL5+qSSz7VgF0JxJH9M= 2021-07-21T03:38:42.710 INFO:teuthology.kill: gibba029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgLbUctGt/HkvFlM+abc+jEzVJIB1mOghLlfjH6c4vOeGo9ot5OkehWqkmpwfzVec28XCvwNHpqUw8mIygSyUEbyPbaXpc7aEJzUMxCjmvoP9rI4PqjtWAQupnxsdsVT86bWBU2WFnvsZ+n8CQbyBJ7DVEwDECcI6szQ7pHhEKSG2YcQVc/17F4FQ5Ojn7KR2buqSJzrhnIRIfGtQGruvKfC+H3UgkmE0uET+cG0YnE0YQom2zwxWSVPWRgHJ4yqSYRJXaRF+xD8drfv/1BZq7nJIaVUKU6Vf04t5P56KKR5w8U5ctRTYs+KH1wrTPyTpJyIWSopbFn3yuIhDCGoN4K1TpIrG0/UdvScDSfBnCmL5MYsqOj9s00yGXmBLEB5vEuSyQToVXWtpcWN6FW4n3m7tkH1XTwSDZrlI9TqMBi0o/92W2GLB8vmNruTznc97jd4GpLaD/mP6QQ1xmRYNJkpk55Dd7+hKrhWvAmTCTXQJdzsfAV0Mgo2m4/8MvJ/M= 2021-07-21T03:38:42.710 INFO:teuthology.kill: gibba035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv/0B1xoiwVEIJbbVdvoYXbS/0vaRfDhjZNsI0dbQzUFFSazgLO8Uc3DX9iOgmbEviVKu9k4z5HHY83VTjkR3PW01W2toi1uX/HCK/CdbGj5xhGtZxhRJY3z0I/AIM1fSQEehQdrVpdrtegh6UP8fXMHGkt/pqzCLEhVD9018yBrnvwQSXf3KMA1hVFKKW79P3kG8gLIDM1PVKUQTzGkbp8TArdU6EZVBhLAJ88t9Xn/GiDVY84+xuRrJm3DG04ZAhyfz8f0KbNWdhFWH8HVNJqZzrS5XdqSnCMrj4NAuY5BdoHT96iON4AAfyl5IyTGlCbB9sDzNIgQH4ZUNqJ/fwUA88cncgd0R6WRv3yGxpD6cOIg7h2q2bxCIGukYefs3StkxuSuYsvm2mzrFWdSkBfFtk5/WNbP1OvMJ0LRedkjxwV3b7Bksl0p5871RmPgx9q4Pj3rhdHpQYz04JUXuz23lz5N7lEoL8BbhwQig3GB6vbCAkLAbrU+mVFa/DzSs= 2021-07-21T03:38:42.711 INFO:teuthology.kill: gibba040.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0WqPkm3Q/OZ0McEUNxaWwAXXgP5GJJngIAEq3IzXo6qEyQJDbJYZiS6Kmo+TE0W70mZ5+RK/0OkeK9b0caI+k7hvihE++FLPlpz5c7IqewDfXrd0CtfAN8+Vae/8R2FsYDnmDVOgdSOLdiWLTC3v0tjRTtjV3eo2IWWM9hsOh9Sxcz1gHf2gHWAoe17Kzk2C9kxeLgOKHC2nFz+W1vA5N0H5KCu8q6Grn4S/yNPhaZj2hj2AS1ogbjzhCZ4vxTaaQ7SxyXNWUL/pZkFyfRfPwkH3MTwfYdzUziPrCrU0w8kSwAIO8pYDYj98pVj4H0wnklPe5e0cnYTehgcjs5KYm5G/Pwt06coitNsTLSaro9FNxZOPgtDoIjEV2m1/Ytf7UAVPKneCF6ADfCLri9+GmsR1xgJqspOCmaP4d5H6zWV0jVhgTlS+EaeVzQNUal77jo60JcDiA/cxHsVydoCMq2qiMEzpwd+OC1v9ltacFAJYxYFymullem2gdE4yu3K0= 2021-07-21T03:38:42.711 INFO:teuthology.kill:2021-07-21 03:38:42,709.709 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:38:42.711 INFO:teuthology.kill:2021-07-21 03:38:42,711.711 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:38:42.712 INFO:teuthology.kill:2021-07-21 03:38:42,712.712 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:38:42.713 INFO:teuthology.kill:2021-07-21 03:38:42,713.713 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:38:42.714 INFO:teuthology.kill:2021-07-21 03:38:42,714.714 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:38:42.776 INFO:teuthology.kill:2021-07-21 03:38:42,775.775 INFO:teuthology.orchestra.console:Power off gibba028 2021-07-21T03:38:42.791 INFO:teuthology.kill:2021-07-21 03:38:42,791.791 INFO:teuthology.orchestra.console:Power off gibba040 2021-07-21T03:38:42.801 INFO:teuthology.kill:2021-07-21 03:38:42,801.801 INFO:teuthology.orchestra.console:Power off gibba029 2021-07-21T03:38:42.811 INFO:teuthology.kill:2021-07-21 03:38:42,811.811 INFO:teuthology.orchestra.console:Power off gibba015 2021-07-21T03:38:42.822 INFO:teuthology.kill:2021-07-21 03:38:42,821.821 INFO:teuthology.orchestra.console:Power off gibba035 2021-07-21T03:38:51.187 INFO:teuthology.kill:2021-07-21 03:38:51,187.187 INFO:teuthology.orchestra.console:Power off for gibba028 completed 2021-07-21T03:38:51.274 INFO:teuthology.kill:2021-07-21 03:38:51,274.274 INFO:teuthology.orchestra.console:Power off for gibba015 completed 2021-07-21T03:38:55.306 INFO:teuthology.kill:2021-07-21 03:38:55,306.306 INFO:teuthology.orchestra.console:Power off for gibba035 completed 2021-07-21T03:38:55.370 INFO:teuthology.kill:2021-07-21 03:38:55,370.370 INFO:teuthology.orchestra.console:Power off for gibba029 completed 2021-07-21T03:38:55.376 INFO:teuthology.kill:2021-07-21 03:38:55,375.375 INFO:teuthology.orchestra.console:Power off for gibba040 completed 2021-07-21T03:38:55.580 INFO:teuthology.task.internal:roles: ubuntu@gibba015.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2021-07-21T03:38:55.581 INFO:teuthology.task.internal:roles: ubuntu@gibba028.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-07-21T03:38:55.581 INFO:teuthology.task.internal:roles: ubuntu@gibba029.front.sepia.ceph.com - ['client.0'] 2021-07-21T03:38:55.581 INFO:teuthology.task.internal:roles: ubuntu@gibba035.front.sepia.ceph.com - ['client.1'] 2021-07-21T03:38:55.582 INFO:teuthology.task.internal:roles: ubuntu@gibba040.front.sepia.ceph.com - ['client.2'] 2021-07-21T03:38:55.582 INFO:teuthology.misc:Compressing logs... 2021-07-21T03:38:55.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-21T03:39:30.123 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_43790980b608e9a71ad69685c3f7e95578734f1b/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_43790980b608e9a71ad69685c3f7e95578734f1b/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_43790980b608e9a71ad69685c3f7e95578734f1b/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_43790980b608e9a71ad69685c3f7e95578734f1b/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_43790980b608e9a71ad69685c3f7e95578734f1b/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_43790980b608e9a71ad69685c3f7e95578734f1b/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_43790980b608e9a71ad69685c3f7e95578734f1b/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-07-21T03:39:30.182 INFO:teuthology.kill:No teuthology processes running 2021-07-21T03:39:31.122 INFO:teuthology.kill:Nuking machines: ['gibba015', 'gibba028', 'gibba029', 'gibba035', 'gibba040'] 2021-07-21T03:39:31.913 INFO:teuthology.kill:2021-07-21 03:39:31,913.913 INFO:teuthology.nuke:targets: 2021-07-21T03:39:31.914 INFO:teuthology.kill: gibba015.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHWL2R0ErOtc2mvnunbyhx1L5+W3q+Q+iYyI1Plzhv8u9yGXUakA/D6R3bmGI7SZdHEyzLjFVLOtdz35P+EGFgtHQtOPSubQRu2ZwiMibvzLaj2e7TXQ/6cvhJ0ouse0QXng5fMNrP6s28bqWy8G3BPeWkprVOk6OElu4nuHw2d8KxQjkp5uY4CunZhXahSBH01M9Yv5rXyS2MDqQtSzaZ3ptGxZDhqDd5LwMfQ1YU2QARbzB/oBV/2XHMVdiFmczQvVpji3cT0piOUNOmb+cSG86epMLdJJ0+U4DScNMi9BEoUcACw/yEbYRP6j5aBemhMGe/UdlMLh5AWQpOSyugDXLgJ+PDDPIiCSENSekrLIkv9ZVy1qCYyArwhL8bCDoTeDAQKjWfMv/gV4khv7tNaqFzorWHKbbE4aMBDVkID7oinA800pYfzXHlXKPh9O80up/KECO9+RkecuM6Si0oKWblrgWK/Dxe3ZorWlLRyL4HQx7jt23aWyIjHPLWnVM= 2021-07-21T03:39:31.915 INFO:teuthology.kill: gibba028.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzaQs7cSZZFYjlWlpiq6Jbc406g0FbW3nmiB4WzyUuxedn8DJNINZtdeWdTIDKzbM/I+sSQdP8aXdTu+yrvgcdEeCIBE6l6BTlBIytzWwoHzDkGta9MJtMdkk3fHxy6P8YOzM+qFPAYaJ5kMWdH74rZxqQqw+4jvvLJW3RCe+6Bn2LiFKQnIfD/5nAJLWSC+ei/gWnwFiM6zYph1RODpxHgAoA7zj7dP7EnEa6gF+cD4bqbLPDSJjJIicBHFnVzz3inpdd9qQHiamJViFpMKS4hLGwh7uR6voRXbTSeMSHis8eTHXlA9lnd5vtTKdaNimR0fHs9Fa/tCMeoKV7QWyApUk1HItui0g1Ud84rvm7NvK0GPJoD2AzLNXCyWk1GMZ1Ay4GFEM7MkRcDV0KPrQWKUhAfQvZZYmHeV0i3oLCtEIcSnmoGmvu1GPyiGB55guXJwF5IEf5FOxuCriB+icrfT5CkwdeWTH7SCs7fBvs8tQvLL5+qSSz7VgF0JxJH9M= 2021-07-21T03:39:31.915 INFO:teuthology.kill: gibba029.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCgLbUctGt/HkvFlM+abc+jEzVJIB1mOghLlfjH6c4vOeGo9ot5OkehWqkmpwfzVec28XCvwNHpqUw8mIygSyUEbyPbaXpc7aEJzUMxCjmvoP9rI4PqjtWAQupnxsdsVT86bWBU2WFnvsZ+n8CQbyBJ7DVEwDECcI6szQ7pHhEKSG2YcQVc/17F4FQ5Ojn7KR2buqSJzrhnIRIfGtQGruvKfC+H3UgkmE0uET+cG0YnE0YQom2zwxWSVPWRgHJ4yqSYRJXaRF+xD8drfv/1BZq7nJIaVUKU6Vf04t5P56KKR5w8U5ctRTYs+KH1wrTPyTpJyIWSopbFn3yuIhDCGoN4K1TpIrG0/UdvScDSfBnCmL5MYsqOj9s00yGXmBLEB5vEuSyQToVXWtpcWN6FW4n3m7tkH1XTwSDZrlI9TqMBi0o/92W2GLB8vmNruTznc97jd4GpLaD/mP6QQ1xmRYNJkpk55Dd7+hKrhWvAmTCTXQJdzsfAV0Mgo2m4/8MvJ/M= 2021-07-21T03:39:31.915 INFO:teuthology.kill: gibba035.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCv/0B1xoiwVEIJbbVdvoYXbS/0vaRfDhjZNsI0dbQzUFFSazgLO8Uc3DX9iOgmbEviVKu9k4z5HHY83VTjkR3PW01W2toi1uX/HCK/CdbGj5xhGtZxhRJY3z0I/AIM1fSQEehQdrVpdrtegh6UP8fXMHGkt/pqzCLEhVD9018yBrnvwQSXf3KMA1hVFKKW79P3kG8gLIDM1PVKUQTzGkbp8TArdU6EZVBhLAJ88t9Xn/GiDVY84+xuRrJm3DG04ZAhyfz8f0KbNWdhFWH8HVNJqZzrS5XdqSnCMrj4NAuY5BdoHT96iON4AAfyl5IyTGlCbB9sDzNIgQH4ZUNqJ/fwUA88cncgd0R6WRv3yGxpD6cOIg7h2q2bxCIGukYefs3StkxuSuYsvm2mzrFWdSkBfFtk5/WNbP1OvMJ0LRedkjxwV3b7Bksl0p5871RmPgx9q4Pj3rhdHpQYz04JUXuz23lz5N7lEoL8BbhwQig3GB6vbCAkLAbrU+mVFa/DzSs= 2021-07-21T03:39:31.915 INFO:teuthology.kill: gibba040.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0WqPkm3Q/OZ0McEUNxaWwAXXgP5GJJngIAEq3IzXo6qEyQJDbJYZiS6Kmo+TE0W70mZ5+RK/0OkeK9b0caI+k7hvihE++FLPlpz5c7IqewDfXrd0CtfAN8+Vae/8R2FsYDnmDVOgdSOLdiWLTC3v0tjRTtjV3eo2IWWM9hsOh9Sxcz1gHf2gHWAoe17Kzk2C9kxeLgOKHC2nFz+W1vA5N0H5KCu8q6Grn4S/yNPhaZj2hj2AS1ogbjzhCZ4vxTaaQ7SxyXNWUL/pZkFyfRfPwkH3MTwfYdzUziPrCrU0w8kSwAIO8pYDYj98pVj4H0wnklPe5e0cnYTehgcjs5KYm5G/Pwt06coitNsTLSaro9FNxZOPgtDoIjEV2m1/Ytf7UAVPKneCF6ADfCLri9+GmsR1xgJqspOCmaP4d5H6zWV0jVhgTlS+EaeVzQNUal77jo60JcDiA/cxHsVydoCMq2qiMEzpwd+OC1v9ltacFAJYxYFymullem2gdE4yu3K0= 2021-07-21T03:39:31.937 INFO:teuthology.kill:2021-07-21 03:39:31,937.937 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:39:31.942 INFO:teuthology.kill:2021-07-21 03:39:31,941.941 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:39:31.943 INFO:teuthology.kill:2021-07-21 03:39:31,943.943 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:39:31.944 INFO:teuthology.kill:2021-07-21 03:39:31,944.944 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:39:31.945 INFO:teuthology.kill:2021-07-21 03:39:31,945.945 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:39:31.999 INFO:teuthology.kill:2021-07-21 03:39:31,998.998 INFO:teuthology.orchestra.console:Power off gibba035 2021-07-21T03:39:32.012 INFO:teuthology.kill:2021-07-21 03:39:32,012.012 INFO:teuthology.orchestra.console:Power off gibba040 2021-07-21T03:39:32.022 INFO:teuthology.kill:2021-07-21 03:39:32,022.022 INFO:teuthology.orchestra.console:Power off gibba028 2021-07-21T03:39:32.034 INFO:teuthology.kill:2021-07-21 03:39:32,033.033 INFO:teuthology.orchestra.console:Power off gibba029 2021-07-21T03:39:32.044 INFO:teuthology.kill:2021-07-21 03:39:32,044.044 INFO:teuthology.orchestra.console:Power off gibba015 2021-07-21T03:39:32.179 INFO:teuthology.kill:2021-07-21 03:39:32,179.179 INFO:teuthology.orchestra.console:Power off for gibba028 completed 2021-07-21T03:39:32.179 INFO:teuthology.kill:2021-07-21 03:39:32,179.179 INFO:teuthology.orchestra.console:Power off for gibba035 completed 2021-07-21T03:39:32.183 INFO:teuthology.kill:2021-07-21 03:39:32,182.182 INFO:teuthology.orchestra.console:Power off for gibba040 completed 2021-07-21T03:39:32.190 INFO:teuthology.kill:2021-07-21 03:39:32,190.190 INFO:teuthology.orchestra.console:Power off for gibba015 completed 2021-07-21T03:39:32.194 INFO:teuthology.kill:2021-07-21 03:39:32,194.194 INFO:teuthology.orchestra.console:Power off for gibba029 completed 2021-07-21T03:39:32.328 INFO:teuthology.kill:2021-07-21 03:39:32,327.327 INFO:teuthology.lock.ops:unlocked gibba028.front.sepia.ceph.com 2021-07-21T03:39:32.330 INFO:teuthology.kill:2021-07-21 03:39:32,330.330 ERROR:teuthology.lock.ops:failed to unlock gibba035.front.sepia.ceph.com. reason: 500 2021-07-21T03:39:32.331 INFO:teuthology.kill:2021-07-21 03:39:32,331.331 ERROR:teuthology.lock.ops:failed to unlock gibba040.front.sepia.ceph.com. reason: 500 2021-07-21T03:39:32.335 INFO:teuthology.kill:2021-07-21 03:39:32,335.335 INFO:teuthology.lock.ops:unlocked gibba015.front.sepia.ceph.com 2021-07-21T03:39:32.336 INFO:teuthology.kill:2021-07-21 03:39:32,336.336 ERROR:teuthology.lock.ops:failed to unlock gibba029.front.sepia.ceph.com. reason: 500 2021-07-21T03:39:32.413 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/teuthology-2021-07-16_04:10:03-fs-octopus-distro-basic-gibba/jobs/ failed with status 500: Internal Server Error

Internal Server Error

2021-07-21T03:41:33.594 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-07-21T03:41:33.728 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-07-21T03:41:33.729 INFO:teuthology.nuke:Checking targets against current locks 2021-07-21T03:41:33.814 INFO:teuthology.nuke:Not nuking gibba028.front.sepia.ceph.com because description doesn't match 2021-07-21T03:41:33.814 INFO:teuthology.nuke:Not nuking gibba015.front.sepia.ceph.com because description doesn't match 2021-07-21T03:41:33.835 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:41:33.839 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:41:33.841 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-21T03:41:33.895 INFO:teuthology.orchestra.console:Power off gibba040 2021-07-21T03:41:33.908 INFO:teuthology.orchestra.console:Power off gibba029 2021-07-21T03:41:33.918 INFO:teuthology.orchestra.console:Power off gibba035 2021-07-21T03:41:34.048 INFO:teuthology.orchestra.console:Power off for gibba040 completed 2021-07-21T03:41:34.054 INFO:teuthology.orchestra.console:Power off for gibba029 completed 2021-07-21T03:41:34.062 INFO:teuthology.orchestra.console:Power off for gibba035 completed 2021-07-21T03:41:34.189 ERROR:teuthology.lock.ops:failed to unlock gibba029.front.sepia.ceph.com. reason: 500 2021-07-21T03:41:34.191 INFO:teuthology.lock.ops:unlocked gibba040.front.sepia.ceph.com 2021-07-21T03:41:34.208 INFO:teuthology.lock.ops:unlocked gibba035.front.sepia.ceph.com