2021-03-29T02:18:18.894 INFO:root:teuthology version: 1.1.0-b9656917 2021-03-29T02:18:18.986 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba015.front.sepia.ceph.com' reimaging 2021-03-29T02:18:18.987 INFO:teuthology.lock.ops:Updating [ubuntu@gibba015.front.sepia.ceph.com]: reset os type and version on server 2021-03-29T02:18:18.988 INFO:teuthology.lock.ops:Updating gibba015.front.sepia.ceph.com on lock server 2021-03-29T02:18:19.001 INFO:teuthology.lock.ops:Node 'ubuntu@gibba015.front.sepia.ceph.com' reimaging is complete 2021-03-29T02:18:19.001 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba016.front.sepia.ceph.com' reimaging 2021-03-29T02:18:19.002 INFO:teuthology.lock.ops:Updating [ubuntu@gibba016.front.sepia.ceph.com]: reset os type and version on server 2021-03-29T02:18:19.002 INFO:teuthology.lock.ops:Updating gibba016.front.sepia.ceph.com on lock server 2021-03-29T02:18:19.015 INFO:teuthology.lock.ops:Node 'ubuntu@gibba016.front.sepia.ceph.com' reimaging is complete 2021-03-29T02:18:19.194 INFO:teuthology.provision.fog.gibba015:Scheduling deploy of ubuntu 20.04 2021-03-29T02:18:19.206 INFO:teuthology.provision.fog.gibba016:Scheduling deploy of ubuntu 20.04 2021-03-29T02:18:19.518 INFO:teuthology.orchestra.console:Power off gibba015 2021-03-29T02:18:19.554 INFO:teuthology.orchestra.console:Power off gibba016 2021-03-29T02:18:19.666 INFO:teuthology.orchestra.console:Power off for gibba015 completed 2021-03-29T02:18:19.701 INFO:teuthology.orchestra.console:Power off for gibba016 completed 2021-03-29T02:18:19.767 INFO:teuthology.orchestra.console:Power on gibba015 2021-03-29T02:18:19.802 INFO:teuthology.orchestra.console:Power on gibba016 2021-03-29T02:18:28.141 INFO:teuthology.orchestra.console:Power on for gibba015 completed 2021-03-29T02:18:28.174 INFO:teuthology.orchestra.console:Power on for gibba016 completed 2021-03-29T02:18:28.243 INFO:teuthology.provision.fog.gibba015:Waiting for deploy to finish 2021-03-29T02:18:28.276 INFO:teuthology.provision.fog.gibba016:Waiting for deploy to finish 2021-03-29T02:23:13.962 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-29T02:23:14.356 INFO:teuthology.provision.fog.gibba016:Node is ready 2021-03-29T02:23:14.363 INFO:teuthology.orchestra.run.gibba016.stdout:gibba016.front.sepia.ceph.com 2021-03-29T02:23:14.371 INFO:teuthology.orchestra.run.gibba016.stdout:172.21.2.116 gibba016.front.sepia.ceph.com gibba016 2021-03-29T02:23:14.555 INFO:teuthology.provision.fog.gibba016:Deploy complete! 2021-03-29T02:23:18.439 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-03-29T02:23:18.844 INFO:teuthology.provision.fog.gibba015:Node is ready 2021-03-29T02:23:18.893 INFO:teuthology.orchestra.run.gibba015.stdout:gibba015.front.sepia.ceph.com 2021-03-29T02:23:18.897 INFO:teuthology.orchestra.run.gibba015.stdout:172.21.2.115 gibba015.front.sepia.ceph.com gibba015 2021-03-29T02:23:19.067 INFO:teuthology.provision.fog.gibba015:Deploy complete! 2021-03-29T02:23:19.205 INFO:teuthology.lock.ops:Checking gibba015.front.sepia.ceph.com 2021-03-29T02:23:19.206 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-29T02:23:19.221 INFO:teuthology.lock.ops:Checking gibba016.front.sepia.ceph.com 2021-03-29T02:23:19.222 INFO:teuthology.lock.ops:New key found. Updating... 2021-03-29T02:23:19.235 INFO:teuthology.lock.ops:Updating [gibba015.front.sepia.ceph.com]: set os type and version on server 2021-03-29T02:23:19.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-29T02:23:19.401 INFO:teuthology.orchestra.run.gibba015.stdout:x86_64 2021-03-29T02:23:19.409 INFO:teuthology.orchestra.run.gibba015.stdout:NAME="Ubuntu" 2021-03-29T02:23:19.410 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-03-29T02:23:19.410 INFO:teuthology.orchestra.run.gibba015.stdout:ID=ubuntu 2021-03-29T02:23:19.410 INFO:teuthology.orchestra.run.gibba015.stdout:ID_LIKE=debian 2021-03-29T02:23:19.410 INFO:teuthology.orchestra.run.gibba015.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-03-29T02:23:19.411 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION_ID="20.04" 2021-03-29T02:23:19.411 INFO:teuthology.orchestra.run.gibba015.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-29T02:23:19.411 INFO:teuthology.orchestra.run.gibba015.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-29T02:23:19.411 INFO:teuthology.orchestra.run.gibba015.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-29T02:23:19.412 INFO:teuthology.orchestra.run.gibba015.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-29T02:23:19.412 INFO:teuthology.orchestra.run.gibba015.stdout:VERSION_CODENAME=focal 2021-03-29T02:23:19.412 INFO:teuthology.orchestra.run.gibba015.stdout:UBUNTU_CODENAME=focal 2021-03-29T02:23:19.413 INFO:teuthology.lock.ops:Updating gibba015.front.sepia.ceph.com on lock server 2021-03-29T02:23:19.428 INFO:teuthology.lock.ops:Updating [gibba016.front.sepia.ceph.com]: set os type and version on server 2021-03-29T02:23:19.429 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-29T02:23:19.592 INFO:teuthology.orchestra.run.gibba016.stdout:x86_64 2021-03-29T02:23:19.600 INFO:teuthology.orchestra.run.gibba016.stdout:NAME="Ubuntu" 2021-03-29T02:23:19.600 INFO:teuthology.orchestra.run.gibba016.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-03-29T02:23:19.600 INFO:teuthology.orchestra.run.gibba016.stdout:ID=ubuntu 2021-03-29T02:23:19.601 INFO:teuthology.orchestra.run.gibba016.stdout:ID_LIKE=debian 2021-03-29T02:23:19.601 INFO:teuthology.orchestra.run.gibba016.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-03-29T02:23:19.601 INFO:teuthology.orchestra.run.gibba016.stdout:VERSION_ID="20.04" 2021-03-29T02:23:19.601 INFO:teuthology.orchestra.run.gibba016.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-29T02:23:19.602 INFO:teuthology.orchestra.run.gibba016.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-29T02:23:19.602 INFO:teuthology.orchestra.run.gibba016.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-29T02:23:19.602 INFO:teuthology.orchestra.run.gibba016.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-29T02:23:19.602 INFO:teuthology.orchestra.run.gibba016.stdout:VERSION_CODENAME=focal 2021-03-29T02:23:19.602 INFO:teuthology.orchestra.run.gibba016.stdout:UBUNTU_CODENAME=focal 2021-03-29T02:23:19.603 INFO:teuthology.lock.ops:Updating gibba016.front.sepia.ceph.com on lock server 2021-03-29T02:23:19.687 INFO:teuthology.dispatcher.supervisor:Running job 5964586 2021-03-29T02:23:19.687 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-03-14_07:01:02-rados-master-distro-basic-gibba/5964586 --name teuthology-2021-03-14_07:01:02-rados-master-distro-basic-gibba --description rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait mon_election/classic} -- /home/teuthworker/archive/teuthology-2021-03-14_07:01:02-rados-master-distro-basic-gibba/5964586/orig.config.yaml 2021-03-29T02:23:19.691 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-03-14_07:01:02-rados-master-distro-basic-gibba/5964586 2021-03-29T02:23:19.692 INFO:teuthology.dispatcher.supervisor:Job PID: 26296 2021-03-29T02:23:19.692 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-03-29T14:23:56.888 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-03-29T14:23:57.446 INFO:teuthology.kill:Killing Pids: {26296} 2021-03-29T14:23:59.495 INFO:teuthology.kill:Nuking machines: ['gibba015', 'gibba016'] 2021-03-29T14:24:00.487 INFO:teuthology.kill:2021-03-29 14:24:00,486.486 INFO:teuthology.nuke:targets: 2021-03-29T14:24:00.488 INFO:teuthology.kill: gibba015.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWKbcStyxtqaY8oB53On10eo2GBppPUY0WJ2cW5yxjNEw/EYdK6SYi1xkHf09BvNRuvvRAv2QbjB8ou8941mmxv2alt7JMkLtAuFnT/tXN0RsI5IhA8tg5du9chuzXtVskuvYhggMJfqU4Ihe9zQle72h6gNIAg5ET5CDUH824vy4b71BeevMRnYJ+yLbYcf6m45Wd2W/MQH/MjUvPd44H6woZ6Gh88AGfnWfdfUDNgPeirUSFIhNqN8MAHJnlWt+C/zopb7Qt6paUIi51KIuVpXJLHs5hiZqHEIkUxrdImaV8wzFO8tMHaATp5fyL6hpOMgdRjtgs7g4VKm9MHQwmOZWalheuSnTRra2KK+0EqURaKEf91inGIX4Zt6Q7ia45wzrgKcpFpPbg7g5SKy90C3zaeNlfF0abHfsE8by+eZI58cXLr43l6pLCGjXVzrnr68e+7BcMCweNC/vL61my0sKl1zeEE7y5LjRdBF6r27ce2M40MyVZ7xjaywUQJdM= 2021-03-29T14:24:00.490 INFO:teuthology.kill: gibba016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkU5ZWiR+61ZROX1OdVcWz2JGnd5OP6mk3rI/rw8FUrMcXj90J3cvNtnJB2VDk599KSVXJNceuN/EJQetSV8+SrEAJJ/HX/oBCyNWcHSkKgvbuWEY9qCXaRAcwFIRZdZrPdF+/jLhcrqbxBSFgrQUSpmSZpBLLsoE59g8fg0N/B4K9OlYE1SBwVHwbYmUw5uFKcK/oYRtc1AvWIQ9VXjG1wvX69qP6VlBFec4hZCJD3Iqo5jolZrU6cdAyiYv0Lw3zAERYKEYAJgxODviwSKntVHl23ifSzHkOZIvvk6oeb8cjrDbS9UxgNgCYMvwEey7F8GRC6SiPWtQhwL/Ozp530GQi4F81lFrmq4nPHSCNvEfKiGT1zXNazzOXJqEzoHcTOO6gdqavK4SaDcxE7Mzr4Zn/25YWeWDf7hKbOTY/6LcWSuuf4KiV0Fy27W0G1NYAg82EjfS0xQJeTFOfx8UQdA6+50tw8I7Qbc+sAk3Tv4Ub8YtM7rnBPdJb30H7vmc= 2021-03-29T14:24:00.490 INFO:teuthology.kill:2021-03-29 14:24:00,487.487 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-29T14:24:00.490 INFO:teuthology.kill:2021-03-29 14:24:00,489.489 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-29T14:24:00.788 INFO:teuthology.kill:2021-03-29 14:24:00,787.787 INFO:teuthology.orchestra.console:Power off gibba015 2021-03-29T14:24:00.802 INFO:teuthology.kill:2021-03-29 14:24:00,801.801 INFO:teuthology.orchestra.console:Power off gibba016 2021-03-29T14:24:09.179 INFO:teuthology.kill:2021-03-29 14:24:09,178.178 INFO:teuthology.orchestra.console:Power off for gibba015 completed 2021-03-29T14:24:09.185 INFO:teuthology.kill:2021-03-29 14:24:09,184.184 INFO:teuthology.orchestra.console:Power off for gibba016 completed 2021-03-29T14:24:09.450 INFO:teuthology.task.internal:roles: ubuntu@gibba015.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2021-03-29T14:24:09.460 INFO:teuthology.task.internal:roles: ubuntu@gibba016.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2021-03-29T14:24:09.466 INFO:teuthology.misc:Compressing logs... 2021-03-29T14:24:09.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-29T14:25:09.662 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/misc.py", line 1374, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-03-29T14:25:09.720 INFO:teuthology.kill:No teuthology processes running 2021-03-29T14:25:11.469 INFO:teuthology.kill:Nuking machines: ['gibba015', 'gibba016'] 2021-03-29T14:25:12.315 INFO:teuthology.kill:2021-03-29 14:25:12,314.314 INFO:teuthology.nuke:targets: 2021-03-29T14:25:12.316 INFO:teuthology.kill: gibba015.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWKbcStyxtqaY8oB53On10eo2GBppPUY0WJ2cW5yxjNEw/EYdK6SYi1xkHf09BvNRuvvRAv2QbjB8ou8941mmxv2alt7JMkLtAuFnT/tXN0RsI5IhA8tg5du9chuzXtVskuvYhggMJfqU4Ihe9zQle72h6gNIAg5ET5CDUH824vy4b71BeevMRnYJ+yLbYcf6m45Wd2W/MQH/MjUvPd44H6woZ6Gh88AGfnWfdfUDNgPeirUSFIhNqN8MAHJnlWt+C/zopb7Qt6paUIi51KIuVpXJLHs5hiZqHEIkUxrdImaV8wzFO8tMHaATp5fyL6hpOMgdRjtgs7g4VKm9MHQwmOZWalheuSnTRra2KK+0EqURaKEf91inGIX4Zt6Q7ia45wzrgKcpFpPbg7g5SKy90C3zaeNlfF0abHfsE8by+eZI58cXLr43l6pLCGjXVzrnr68e+7BcMCweNC/vL61my0sKl1zeEE7y5LjRdBF6r27ce2M40MyVZ7xjaywUQJdM= 2021-03-29T14:25:12.316 INFO:teuthology.kill: gibba016.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkU5ZWiR+61ZROX1OdVcWz2JGnd5OP6mk3rI/rw8FUrMcXj90J3cvNtnJB2VDk599KSVXJNceuN/EJQetSV8+SrEAJJ/HX/oBCyNWcHSkKgvbuWEY9qCXaRAcwFIRZdZrPdF+/jLhcrqbxBSFgrQUSpmSZpBLLsoE59g8fg0N/B4K9OlYE1SBwVHwbYmUw5uFKcK/oYRtc1AvWIQ9VXjG1wvX69qP6VlBFec4hZCJD3Iqo5jolZrU6cdAyiYv0Lw3zAERYKEYAJgxODviwSKntVHl23ifSzHkOZIvvk6oeb8cjrDbS9UxgNgCYMvwEey7F8GRC6SiPWtQhwL/Ozp530GQi4F81lFrmq4nPHSCNvEfKiGT1zXNazzOXJqEzoHcTOO6gdqavK4SaDcxE7Mzr4Zn/25YWeWDf7hKbOTY/6LcWSuuf4KiV0Fy27W0G1NYAg82EjfS0xQJeTFOfx8UQdA6+50tw8I7Qbc+sAk3Tv4Ub8YtM7rnBPdJb30H7vmc= 2021-03-29T14:25:12.335 INFO:teuthology.kill:2021-03-29 14:25:12,335.335 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-29T14:25:12.347 INFO:teuthology.kill:2021-03-29 14:25:12,347.347 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-29T14:25:12.416 INFO:teuthology.kill:2021-03-29 14:25:12,416.416 INFO:teuthology.orchestra.console:Power off gibba015 2021-03-29T14:25:12.428 INFO:teuthology.kill:2021-03-29 14:25:12,428.428 INFO:teuthology.orchestra.console:Power off gibba016 2021-03-29T14:25:12.567 INFO:teuthology.kill:2021-03-29 14:25:12,567.567 INFO:teuthology.orchestra.console:Power off for gibba015 completed 2021-03-29T14:25:12.575 INFO:teuthology.kill:2021-03-29 14:25:12,574.574 INFO:teuthology.orchestra.console:Power off for gibba016 completed 2021-03-29T14:25:12.720 INFO:teuthology.kill:2021-03-29 14:25:12,720.720 INFO:teuthology.lock.ops:unlocked gibba015.front.sepia.ceph.com 2021-03-29T14:25:12.737 INFO:teuthology.kill:2021-03-29 14:25:12,737.737 INFO:teuthology.lock.ops:unlocked gibba016.front.sepia.ceph.com 2021-03-29T14:27:12.972 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15