2021-05-24T19:45:55.283 INFO:root:teuthology version: 1.1.0-cc3bac84 2021-05-24T19:45:55.372 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba031.front.sepia.ceph.com' reimaging 2021-05-24T19:45:55.373 INFO:teuthology.lock.ops:Updating [ubuntu@gibba031.front.sepia.ceph.com]: reset os type and version on server 2021-05-24T19:45:55.374 INFO:teuthology.lock.ops:Updating gibba031.front.sepia.ceph.com on lock server 2021-05-24T19:45:55.387 INFO:teuthology.lock.ops:Node 'ubuntu@gibba031.front.sepia.ceph.com' reimaging is complete 2021-05-24T19:45:55.388 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba034.front.sepia.ceph.com' reimaging 2021-05-24T19:45:55.388 INFO:teuthology.lock.ops:Updating [ubuntu@gibba034.front.sepia.ceph.com]: reset os type and version on server 2021-05-24T19:45:55.388 INFO:teuthology.lock.ops:Updating gibba034.front.sepia.ceph.com on lock server 2021-05-24T19:45:55.402 INFO:teuthology.lock.ops:Node 'ubuntu@gibba034.front.sepia.ceph.com' reimaging is complete 2021-05-24T19:45:55.570 INFO:teuthology.provision.fog.gibba031:Scheduling deploy of ubuntu 20.04 2021-05-24T19:45:55.590 INFO:teuthology.provision.fog.gibba034:Scheduling deploy of ubuntu 20.04 2021-05-24T19:45:55.940 INFO:teuthology.orchestra.console:Power off gibba031 2021-05-24T19:45:55.977 INFO:teuthology.orchestra.console:Power off gibba034 2021-05-24T19:46:08.431 INFO:teuthology.orchestra.console:Power off for gibba031 completed 2021-05-24T19:46:08.452 INFO:teuthology.orchestra.console:Power off for gibba034 completed 2021-05-24T19:46:08.533 INFO:teuthology.orchestra.console:Power on gibba031 2021-05-24T19:46:08.553 INFO:teuthology.orchestra.console:Power on gibba034 2021-05-24T19:46:12.811 INFO:teuthology.orchestra.console:Power on for gibba034 completed 2021-05-24T19:46:12.912 INFO:teuthology.provision.fog.gibba034:Waiting for deploy to finish 2021-05-24T19:46:16.910 INFO:teuthology.orchestra.console:Power on for gibba031 completed 2021-05-24T19:46:17.012 INFO:teuthology.provision.fog.gibba031:Waiting for deploy to finish 2021-05-24T19:48:44.503 ERROR:teuthology.orchestra.connection:Error authenticating with gibba034.front.sepia.ceph.com: Authentication failed. 2021-05-24T19:50:57.623 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-24T19:50:57.962 INFO:teuthology.provision.fog.gibba031:Node is ready 2021-05-24T19:50:58.012 INFO:teuthology.orchestra.run.gibba031.stdout:gibba031.front.sepia.ceph.com 2021-05-24T19:50:58.020 INFO:teuthology.orchestra.run.gibba031.stdout:172.21.2.131 gibba031.front.sepia.ceph.com gibba031 2021-05-24T19:50:58.193 INFO:teuthology.provision.fog.gibba031:Deploy complete! 2021-05-24T19:51:06.763 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-24T19:51:07.221 INFO:teuthology.provision.fog.gibba034:Node is ready 2021-05-24T19:51:07.227 INFO:teuthology.orchestra.run.gibba034.stdout:gibba034.front.sepia.ceph.com 2021-05-24T19:51:07.278 INFO:teuthology.orchestra.run.gibba034.stdout:172.21.2.134 gibba034.front.sepia.ceph.com gibba034 2021-05-24T19:51:07.431 INFO:teuthology.provision.fog.gibba034:Deploy complete! 2021-05-24T19:51:07.559 INFO:teuthology.lock.ops:Checking gibba031.front.sepia.ceph.com 2021-05-24T19:51:07.560 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-24T19:51:07.577 INFO:teuthology.lock.ops:Checking gibba034.front.sepia.ceph.com 2021-05-24T19:51:07.577 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-24T19:51:07.593 INFO:teuthology.lock.ops:Updating [gibba031.front.sepia.ceph.com]: set os type and version on server 2021-05-24T19:51:07.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-24T19:51:07.774 INFO:teuthology.orchestra.run.gibba031.stdout:x86_64 2021-05-24T19:51:07.782 INFO:teuthology.orchestra.run.gibba031.stdout:NAME="Ubuntu" 2021-05-24T19:51:07.782 INFO:teuthology.orchestra.run.gibba031.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-05-24T19:51:07.783 INFO:teuthology.orchestra.run.gibba031.stdout:ID=ubuntu 2021-05-24T19:51:07.784 INFO:teuthology.orchestra.run.gibba031.stdout:ID_LIKE=debian 2021-05-24T19:51:07.784 INFO:teuthology.orchestra.run.gibba031.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-05-24T19:51:07.784 INFO:teuthology.orchestra.run.gibba031.stdout:VERSION_ID="20.04" 2021-05-24T19:51:07.785 INFO:teuthology.orchestra.run.gibba031.stdout:HOME_URL="https://www.ubuntu.com/" 2021-05-24T19:51:07.785 INFO:teuthology.orchestra.run.gibba031.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-05-24T19:51:07.785 INFO:teuthology.orchestra.run.gibba031.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-05-24T19:51:07.785 INFO:teuthology.orchestra.run.gibba031.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-05-24T19:51:07.786 INFO:teuthology.orchestra.run.gibba031.stdout:VERSION_CODENAME=focal 2021-05-24T19:51:07.786 INFO:teuthology.orchestra.run.gibba031.stdout:UBUNTU_CODENAME=focal 2021-05-24T19:51:07.787 INFO:teuthology.lock.ops:Updating gibba031.front.sepia.ceph.com on lock server 2021-05-24T19:51:07.803 INFO:teuthology.lock.ops:Updating [gibba034.front.sepia.ceph.com]: set os type and version on server 2021-05-24T19:51:07.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-24T19:51:07.949 INFO:teuthology.orchestra.run.gibba034.stdout:x86_64 2021-05-24T19:51:08.001 INFO:teuthology.orchestra.run.gibba034.stdout:NAME="Ubuntu" 2021-05-24T19:51:08.001 INFO:teuthology.orchestra.run.gibba034.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-05-24T19:51:08.002 INFO:teuthology.orchestra.run.gibba034.stdout:ID=ubuntu 2021-05-24T19:51:08.002 INFO:teuthology.orchestra.run.gibba034.stdout:ID_LIKE=debian 2021-05-24T19:51:08.002 INFO:teuthology.orchestra.run.gibba034.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-05-24T19:51:08.002 INFO:teuthology.orchestra.run.gibba034.stdout:VERSION_ID="20.04" 2021-05-24T19:51:08.003 INFO:teuthology.orchestra.run.gibba034.stdout:HOME_URL="https://www.ubuntu.com/" 2021-05-24T19:51:08.003 INFO:teuthology.orchestra.run.gibba034.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-05-24T19:51:08.003 INFO:teuthology.orchestra.run.gibba034.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-05-24T19:51:08.003 INFO:teuthology.orchestra.run.gibba034.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-05-24T19:51:08.003 INFO:teuthology.orchestra.run.gibba034.stdout:VERSION_CODENAME=focal 2021-05-24T19:51:08.004 INFO:teuthology.orchestra.run.gibba034.stdout:UBUNTU_CODENAME=focal 2021-05-24T19:51:08.004 INFO:teuthology.lock.ops:Updating gibba034.front.sepia.ceph.com on lock server 2021-05-24T19:51:08.090 INFO:teuthology.dispatcher.supervisor:Running job 6117268 2021-05-24T19:51:08.091 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-05-16_07:01:03-rados-master-distro-basic-gibba/6117268 --name teuthology-2021-05-16_07:01:03-rados-master-distro-basic-gibba --description rados/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 1-start 2-services/rgw-ingress 3-final} -- /home/teuthworker/archive/teuthology-2021-05-16_07:01:03-rados-master-distro-basic-gibba/6117268/orig.config.yaml 2021-05-24T19:51:08.098 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-05-16_07:01:03-rados-master-distro-basic-gibba/6117268 2021-05-24T19:51:08.099 INFO:teuthology.dispatcher.supervisor:Job PID: 26294 2021-05-24T19:51:08.102 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-05-25T07:51:23.995 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-05-25T07:51:24.239 INFO:teuthology.kill:Killing Pids: {26294} 2021-05-25T07:51:26.099 INFO:teuthology.kill:Nuking machines: ['gibba031', 'gibba034'] 2021-05-25T07:51:27.097 INFO:teuthology.kill:2021-05-25 07:51:27,097.097 INFO:teuthology.nuke:targets: 2021-05-25T07:51:27.098 INFO:teuthology.kill: gibba031.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0/eL0TMNqoATuoYcOz/FtntszdXoNUoudiLzjUwgsFDJcEgWs6M0eYEOoH0JGiJHxNq6rHqWco5kvGpL9mPuDeGSfScvI6PzEeyZwAR2j4V0H33FukZAgfxakeeBe0HUd2tT8vFCmEDO1e4kmKL1ygbEwWz0wN7KVqnaqc5ZjhpEj7Gr8qXMpQ8J89SOoIK3oYsMNS0xf2pAuTEjrFWXaoUJyubi+8nCAVyHru0rz2M2AF2pswKuOUNCL9CIcdbVuG3+hCpP7CPeEBeyXmYm7/WRHsX+I0ZT0qU3U6309wQA3eP98Bis3G9iE/TKdzxI48rdkeHpXT8qBZZPuJyjvxCl/+j/Q9qOMHb03ETgIhEWytkqtcgE1vEco0zyt4AQYMyvZRFcWCfJ2vw9juc3yaDf+elbbG2luSxS5RA/TP/Q8wr2pVvJFLeX4gB19g45ySNmWhxl7p9V5t7JJx8l5E20odMlksfgV0SDgD/RaQ/VpSPjcHFzg3ZzAxQ1a7Q8= 2021-05-25T07:51:27.099 INFO:teuthology.kill: gibba034.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWa7xbQV1oq9ZlLV6+xKLcpVPkRkhM4fQzCJn9ACZYC16v+O0/KiydFiTnkUnx7aNQdb5yK5/ed/NXM6Q1J9Z+ipuyK5NlUWRYrH16gaGCu/rhYFSMPxJF1Kqqm7cGTfCOUveBlmjyb0v+VnFBOfcxhTotbobTiuk7Cqw4j6w+vXEe4sX8EwM9X3CoOie1g7FfX0r0khGcVjSu/0Fs1yzN4otcTPWst2OojZkM+JJthGdVu7rQVIia4r+6D6VE4oOCmq2Vvn99F1A8+5PY5gG4Dn3b2XlNl6oCv53YVwuC8Pe/x9Hk7DBWGDjgAWiG9U/q7XgFZ0xv0Xek0ob6I9xi+tOR0Z6kewE7YUPhjd4zOKmqBmRwHkioFqRZmdR/mJgFYs7hGy1h2G3uFl6LRTXquOOT+knEHkKdeRfUlfQEdFKAtGigKFVYlidIj7Du0CK8DxScMM21MwIAyZDxIPw9tztdgoNzixqLKXlRIQuWxxcja+eCI+rOsnnSeO3Atrk= 2021-05-25T07:51:27.099 INFO:teuthology.kill:2021-05-25 07:51:27,099.099 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-25T07:51:27.101 INFO:teuthology.kill:2021-05-25 07:51:27,101.101 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-25T07:51:27.157 INFO:teuthology.kill:2021-05-25 07:51:27,156.156 INFO:teuthology.orchestra.console:Power off gibba034 2021-05-25T07:51:27.170 INFO:teuthology.kill:2021-05-25 07:51:27,170.170 INFO:teuthology.orchestra.console:Power off gibba031 2021-05-25T07:51:39.656 INFO:teuthology.kill:2021-05-25 07:51:39,656.656 INFO:teuthology.orchestra.console:Power off for gibba031 completed 2021-05-25T07:51:39.719 INFO:teuthology.kill:2021-05-25 07:51:39,719.719 INFO:teuthology.orchestra.console:Power off for gibba034 completed 2021-05-25T07:51:39.924 INFO:teuthology.task.internal:roles: ubuntu@gibba031.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2021-05-25T07:51:39.925 INFO:teuthology.task.internal:roles: ubuntu@gibba034.front.sepia.ceph.com - ['host.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2021-05-25T07:51:39.926 INFO:teuthology.misc:Compressing logs... 2021-05-25T07:51:39.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-25T07:51:58.411 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-05-25T07:51:58.457 INFO:teuthology.kill:No teuthology processes running 2021-05-25T07:51:59.510 INFO:teuthology.kill:Nuking machines: ['gibba031', 'gibba034'] 2021-05-25T07:52:00.488 INFO:teuthology.kill:2021-05-25 07:52:00,487.487 INFO:teuthology.nuke:targets: 2021-05-25T07:52:00.489 INFO:teuthology.kill: gibba031.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0/eL0TMNqoATuoYcOz/FtntszdXoNUoudiLzjUwgsFDJcEgWs6M0eYEOoH0JGiJHxNq6rHqWco5kvGpL9mPuDeGSfScvI6PzEeyZwAR2j4V0H33FukZAgfxakeeBe0HUd2tT8vFCmEDO1e4kmKL1ygbEwWz0wN7KVqnaqc5ZjhpEj7Gr8qXMpQ8J89SOoIK3oYsMNS0xf2pAuTEjrFWXaoUJyubi+8nCAVyHru0rz2M2AF2pswKuOUNCL9CIcdbVuG3+hCpP7CPeEBeyXmYm7/WRHsX+I0ZT0qU3U6309wQA3eP98Bis3G9iE/TKdzxI48rdkeHpXT8qBZZPuJyjvxCl/+j/Q9qOMHb03ETgIhEWytkqtcgE1vEco0zyt4AQYMyvZRFcWCfJ2vw9juc3yaDf+elbbG2luSxS5RA/TP/Q8wr2pVvJFLeX4gB19g45ySNmWhxl7p9V5t7JJx8l5E20odMlksfgV0SDgD/RaQ/VpSPjcHFzg3ZzAxQ1a7Q8= 2021-05-25T07:52:00.489 INFO:teuthology.kill: gibba034.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWa7xbQV1oq9ZlLV6+xKLcpVPkRkhM4fQzCJn9ACZYC16v+O0/KiydFiTnkUnx7aNQdb5yK5/ed/NXM6Q1J9Z+ipuyK5NlUWRYrH16gaGCu/rhYFSMPxJF1Kqqm7cGTfCOUveBlmjyb0v+VnFBOfcxhTotbobTiuk7Cqw4j6w+vXEe4sX8EwM9X3CoOie1g7FfX0r0khGcVjSu/0Fs1yzN4otcTPWst2OojZkM+JJthGdVu7rQVIia4r+6D6VE4oOCmq2Vvn99F1A8+5PY5gG4Dn3b2XlNl6oCv53YVwuC8Pe/x9Hk7DBWGDjgAWiG9U/q7XgFZ0xv0Xek0ob6I9xi+tOR0Z6kewE7YUPhjd4zOKmqBmRwHkioFqRZmdR/mJgFYs7hGy1h2G3uFl6LRTXquOOT+knEHkKdeRfUlfQEdFKAtGigKFVYlidIj7Du0CK8DxScMM21MwIAyZDxIPw9tztdgoNzixqLKXlRIQuWxxcja+eCI+rOsnnSeO3Atrk= 2021-05-25T07:52:00.508 INFO:teuthology.kill:2021-05-25 07:52:00,507.507 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-25T07:52:00.510 INFO:teuthology.kill:2021-05-25 07:52:00,510.510 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-25T07:52:00.563 INFO:teuthology.kill:2021-05-25 07:52:00,563.563 INFO:teuthology.orchestra.console:Power off gibba031 2021-05-25T07:52:00.578 INFO:teuthology.kill:2021-05-25 07:52:00,577.577 INFO:teuthology.orchestra.console:Power off gibba034 2021-05-25T07:52:00.710 INFO:teuthology.kill:2021-05-25 07:52:00,710.710 INFO:teuthology.orchestra.console:Power off for gibba031 completed 2021-05-25T07:52:00.723 INFO:teuthology.kill:2021-05-25 07:52:00,722.722 INFO:teuthology.orchestra.console:Power off for gibba034 completed 2021-05-25T07:52:00.853 INFO:teuthology.kill:2021-05-25 07:52:00,853.853 INFO:teuthology.lock.ops:unlocked gibba031.front.sepia.ceph.com 2021-05-25T07:52:00.857 INFO:teuthology.kill:2021-05-25 07:52:00,857.857 ERROR:teuthology.lock.ops:failed to unlock gibba034.front.sepia.ceph.com. reason: 500 2021-05-25T07:54:01.032 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-05-25T07:54:01.112 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-05-25T07:54:01.112 INFO:teuthology.nuke:Checking targets against current locks 2021-05-25T07:54:01.195 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/dispatcher/supervisor.py", line 59, in main verbose File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/dispatcher/supervisor.py", line 152, in run_job unlock_targets(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/dispatcher/supervisor.py", line 196, in unlock_targets nuke(fake_ctx, True) File "/home/teuthworker/src/git.ceph.com_git_teuthology_cc3bac844fd368a871b19f94cc5d520695b86d44/teuthology/nuke/__init__.py", line 250, in nuke if ctx.name not in lock['description']: TypeError: argument of type 'NoneType' is not iterable