2021-06-22T21:15:46.460 INFO:root:teuthology version: 1.1.0-f359b10d
2021-06-22T21:15:46.581 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba013.front.sepia.ceph.com' reimaging
2021-06-22T21:15:46.582 INFO:teuthology.lock.ops:Updating [ubuntu@gibba013.front.sepia.ceph.com]: reset os type and version on server
2021-06-22T21:15:46.583 INFO:teuthology.lock.ops:Updating gibba013.front.sepia.ceph.com on lock server
2021-06-22T21:15:46.598 INFO:teuthology.lock.ops:Node 'ubuntu@gibba013.front.sepia.ceph.com' reimaging is complete
2021-06-22T21:15:46.599 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba022.front.sepia.ceph.com' reimaging
2021-06-22T21:15:46.599 INFO:teuthology.lock.ops:Updating [ubuntu@gibba022.front.sepia.ceph.com]: reset os type and version on server
2021-06-22T21:15:46.599 INFO:teuthology.lock.ops:Updating gibba022.front.sepia.ceph.com on lock server
2021-06-22T21:15:46.614 INFO:teuthology.lock.ops:Node 'ubuntu@gibba022.front.sepia.ceph.com' reimaging is complete
2021-06-22T21:15:46.804 INFO:teuthology.provision.fog.gibba013:Scheduling deploy of ubuntu 20.04
2021-06-22T21:15:46.819 INFO:teuthology.provision.fog.gibba022:Scheduling deploy of ubuntu 20.04
2021-06-22T21:15:47.172 INFO:teuthology.orchestra.console:Power off gibba013
2021-06-22T21:15:47.189 INFO:teuthology.orchestra.console:Power off gibba022
2021-06-22T21:15:47.326 INFO:teuthology.orchestra.console:Power off for gibba013 completed
2021-06-22T21:15:47.338 INFO:teuthology.orchestra.console:Power off for gibba022 completed
2021-06-22T21:15:47.428 INFO:teuthology.orchestra.console:Power on gibba013
2021-06-22T21:15:47.439 INFO:teuthology.orchestra.console:Power on gibba022
2021-06-22T21:15:55.807 INFO:teuthology.orchestra.console:Power on for gibba013 completed
2021-06-22T21:15:55.811 INFO:teuthology.orchestra.console:Power on for gibba022 completed
2021-06-22T21:15:55.908 INFO:teuthology.provision.fog.gibba013:Waiting for deploy to finish
2021-06-22T21:15:55.912 INFO:teuthology.provision.fog.gibba022:Waiting for deploy to finish
2021-06-22T21:20:35.931 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-06-22T21:20:36.304 INFO:teuthology.provision.fog.gibba013:Node is ready
2021-06-22T21:20:36.352 INFO:teuthology.orchestra.run.gibba013.stdout:gibba013.front.sepia.ceph.com
2021-06-22T21:20:36.361 INFO:teuthology.orchestra.run.gibba013.stdout:172.21.2.113 gibba013.front.sepia.ceph.com gibba013
2021-06-22T21:20:36.440 INFO:teuthology.provision.fog.gibba013:Deploy complete!
2021-06-22T21:20:46.330 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-06-22T21:20:46.742 INFO:teuthology.provision.fog.gibba022:Node is ready
2021-06-22T21:20:46.749 INFO:teuthology.orchestra.run.gibba022.stdout:gibba022.front.sepia.ceph.com
2021-06-22T21:20:46.757 INFO:teuthology.orchestra.run.gibba022.stdout:172.21.2.122 gibba022.front.sepia.ceph.com gibba022
2021-06-22T21:20:46.898 INFO:teuthology.provision.fog.gibba022:Deploy complete!
2021-06-22T21:20:47.022 INFO:teuthology.lock.ops:Checking gibba013.front.sepia.ceph.com
2021-06-22T21:20:47.023 INFO:teuthology.lock.ops:New key found. Updating...
2021-06-22T21:20:47.041 INFO:teuthology.lock.ops:Checking gibba022.front.sepia.ceph.com
2021-06-22T21:20:47.041 INFO:teuthology.lock.ops:New key found. Updating...
2021-06-22T21:20:47.056 INFO:teuthology.lock.ops:Updating [gibba013.front.sepia.ceph.com]: set os type and version on server
2021-06-22T21:20:47.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-22T21:20:47.182 INFO:teuthology.orchestra.run.gibba013.stdout:x86_64
2021-06-22T21:20:47.186 INFO:teuthology.orchestra.run.gibba013.stdout:NAME="Ubuntu"
2021-06-22T21:20:47.187 INFO:teuthology.orchestra.run.gibba013.stdout:VERSION="20.04.2 LTS (Focal Fossa)"
2021-06-22T21:20:47.187 INFO:teuthology.orchestra.run.gibba013.stdout:ID=ubuntu
2021-06-22T21:20:47.187 INFO:teuthology.orchestra.run.gibba013.stdout:ID_LIKE=debian
2021-06-22T21:20:47.187 INFO:teuthology.orchestra.run.gibba013.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS"
2021-06-22T21:20:47.187 INFO:teuthology.orchestra.run.gibba013.stdout:VERSION_ID="20.04"
2021-06-22T21:20:47.188 INFO:teuthology.orchestra.run.gibba013.stdout:HOME_URL="https://www.ubuntu.com/"
2021-06-22T21:20:47.188 INFO:teuthology.orchestra.run.gibba013.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-06-22T21:20:47.188 INFO:teuthology.orchestra.run.gibba013.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-06-22T21:20:47.188 INFO:teuthology.orchestra.run.gibba013.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-06-22T21:20:47.189 INFO:teuthology.orchestra.run.gibba013.stdout:VERSION_CODENAME=focal
2021-06-22T21:20:47.189 INFO:teuthology.orchestra.run.gibba013.stdout:UBUNTU_CODENAME=focal
2021-06-22T21:20:47.190 INFO:teuthology.lock.ops:Updating gibba013.front.sepia.ceph.com on lock server
2021-06-22T21:20:47.206 INFO:teuthology.lock.ops:Updating [gibba022.front.sepia.ceph.com]: set os type and version on server
2021-06-22T21:20:47.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-22T21:20:47.352 INFO:teuthology.orchestra.run.gibba022.stdout:x86_64
2021-06-22T21:20:47.360 INFO:teuthology.orchestra.run.gibba022.stdout:NAME="Ubuntu"
2021-06-22T21:20:47.360 INFO:teuthology.orchestra.run.gibba022.stdout:VERSION="20.04.2 LTS (Focal Fossa)"
2021-06-22T21:20:47.360 INFO:teuthology.orchestra.run.gibba022.stdout:ID=ubuntu
2021-06-22T21:20:47.361 INFO:teuthology.orchestra.run.gibba022.stdout:ID_LIKE=debian
2021-06-22T21:20:47.361 INFO:teuthology.orchestra.run.gibba022.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS"
2021-06-22T21:20:47.361 INFO:teuthology.orchestra.run.gibba022.stdout:VERSION_ID="20.04"
2021-06-22T21:20:47.361 INFO:teuthology.orchestra.run.gibba022.stdout:HOME_URL="https://www.ubuntu.com/"
2021-06-22T21:20:47.361 INFO:teuthology.orchestra.run.gibba022.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2021-06-22T21:20:47.362 INFO:teuthology.orchestra.run.gibba022.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2021-06-22T21:20:47.362 INFO:teuthology.orchestra.run.gibba022.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2021-06-22T21:20:47.362 INFO:teuthology.orchestra.run.gibba022.stdout:VERSION_CODENAME=focal
2021-06-22T21:20:47.362 INFO:teuthology.orchestra.run.gibba022.stdout:UBUNTU_CODENAME=focal
2021-06-22T21:20:47.363 INFO:teuthology.lock.ops:Updating gibba022.front.sepia.ceph.com on lock server
2021-06-22T21:20:47.434 INFO:teuthology.dispatcher.supervisor:Running job 6143056
2021-06-22T21:20:47.435 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-05-30_07:01:02-rados-master-distro-basic-gibba/6143056 --name teuthology-2021-05-30_07:01:02-rados-master-distro-basic-gibba --description rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait mon_election/classic} -- /home/teuthworker/archive/teuthology-2021-05-30_07:01:02-rados-master-distro-basic-gibba/6143056/orig.config.yaml
2021-06-22T21:20:47.438 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-05-30_07:01:02-rados-master-distro-basic-gibba/6143056
2021-06-22T21:20:47.439 INFO:teuthology.dispatcher.supervisor:Job PID: 12658
2021-06-22T21:20:47.439 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2021-06-22T22:50:53.083 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/teuthology-2021-05-30_07:01:02-rados-master-distro-basic-gibba/jobs/ failed with status 500:
Internal Server Error
Internal Server Error
2021-06-23T07:11:23.950 ERROR:teuthology.report:POST to http://paddles.front.sepia.ceph.com/runs/teuthology-2021-05-30_07:01:02-rados-master-distro-basic-gibba/jobs/ failed with status 500:
Internal Server Error
Internal Server Error
2021-06-23T09:21:32.852 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing...
2021-06-23T09:21:33.026 INFO:teuthology.kill:Killing Pids: {12658}
2021-06-23T09:21:34.010 INFO:teuthology.kill:Nuking machines: ['gibba013', 'gibba022']
2021-06-23T09:21:34.848 INFO:teuthology.kill:2021-06-23 09:21:34,848.848 INFO:teuthology.nuke:targets:
2021-06-23T09:21:34.849 INFO:teuthology.kill: gibba013.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRzu2zTi9jVxJErGIYf6KnHe455FKc/HvPGuCXV0bCcwVT1fhRlr6d5crkns7I5tdPSSAZKy5EJGGZvg4OWLZj3VAb7EH+fzRL+Izvh9Bgt74TBeD0AgOQCQV9r/9s815Fj87SvFpgVt9kIt8UKowovVlp1PsZsCmUy4icXviDPlHj7XYBnztuz9ijCbyiMzUXmFhtZH5d4/By0vA6HIWyK6cgicTTcJCzoG0tYk+/D9siCEKYakB51US1MBL6x4X4WyisNvjb2TIfWhyvp3nTjVZ4CLyMc3KvlBrjpuVffvpAWjAYG0V5FgpoMz6fDLyubOa/AAlDpIz+xO5t9mNp93TRCt7h5h6K2osVNTeteVCOkvkOI3wQbenmZfYTwUpzVx3W9yKhN1xIXFs2M3Ao6Zn1voKVnJpw8urZX4OEsfv5UNAgvAtRWq3SehX6l1H+CQEXK0B0M3XprbcKNJRzKMnEWPzC80XC5azf9jIdAHlG7iFrPxAN+H2fhmRCZx0=
2021-06-23T09:21:34.849 INFO:teuthology.kill: gibba022.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEWTwTAXLYeBTGul4N1FS9V6bhl25Qz9xZdwijYI+ctyEUXM380kq2kUFS2uKnMHGupQUIMT/0+l2T93kEpCJfvWjjcALlUXlprM8pkLTGW+ief7Hj7JfU8wwvwurUAX6hRsColgVdqQWdfpc6HINblBklHgTOPUk6hDBuxjrJSuSislxqwJfvWfYustI7zeaTCnljlkjCMQ7ueXf9agPQbGws/RnQr+ffkFGloKrHOAQOA8O+KEGTwFghc/jSKHnOLw/anprnFbsI1l4QkgXUwyXEYwe1xOo6hm/e0d4lYaQe0/Vm6ARll+hC1wzeuzZ7gi3ZOMlUF/+fKvXX9tX57F2tQhPioVj+pQpukbQL8EOYgh8ujM4BB5CFK+Nun0mJSgZvemNv7gpJH97/QRMHIhgX9Udku4Kxmm5jb3w4LfNfxIJ4OAhulwzO/ce+CeNhMXUcmtwPjjDF+r2xbKWqMxmkl1+z3tOCElHsnDq8YPvZK0utEiK+OIeW2cw6erM=
2021-06-23T09:21:34.850 INFO:teuthology.kill:2021-06-23 09:21:34,848.848 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-23T09:21:34.850 INFO:teuthology.kill:2021-06-23 09:21:34,850.850 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-23T09:21:34.905 INFO:teuthology.kill:2021-06-23 09:21:34,905.905 INFO:teuthology.orchestra.console:Power off gibba022
2021-06-23T09:21:34.917 INFO:teuthology.kill:2021-06-23 09:21:34,917.917 INFO:teuthology.orchestra.console:Power off gibba013
2021-06-23T09:21:43.291 INFO:teuthology.kill:2021-06-23 09:21:43,290.290 INFO:teuthology.orchestra.console:Power off for gibba022 completed
2021-06-23T09:21:43.298 INFO:teuthology.kill:2021-06-23 09:21:43,298.298 INFO:teuthology.orchestra.console:Power off for gibba013 completed
2021-06-23T09:21:43.494 INFO:teuthology.task.internal:roles: ubuntu@gibba013.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a']
2021-06-23T09:21:43.495 INFO:teuthology.task.internal:roles: ubuntu@gibba022.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b']
2021-06-23T09:21:43.495 INFO:teuthology.misc:Compressing logs...
2021-06-23T09:21:43.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2021-06-23T09:22:18.187 ERROR:teuthology.dispatcher.supervisor:Could not save logs
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog
teuth_config.archive_base, job_config)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives
compress_logs(ctx, log_path)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1378, in compress_logs
wait=False,
File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in run
return [remote.run(**kwargs) for remote in remotes]
File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/cluster.py", line 64, in
return [remote.run(**kwargs) for remote in remotes]
File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/remote.py", line 509, in run
r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/orchestra/run.py", line 434, in run
transport = client.get_transport()
AttributeError: 'NoneType' object has no attribute 'get_transport'
2021-06-23T09:22:18.224 INFO:teuthology.kill:No teuthology processes running
2021-06-23T09:22:19.270 INFO:teuthology.kill:Nuking machines: ['gibba013', 'gibba022']
2021-06-23T09:22:20.171 INFO:teuthology.kill:2021-06-23 09:22:20,171.171 INFO:teuthology.nuke:targets:
2021-06-23T09:22:20.172 INFO:teuthology.kill: gibba013.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRzu2zTi9jVxJErGIYf6KnHe455FKc/HvPGuCXV0bCcwVT1fhRlr6d5crkns7I5tdPSSAZKy5EJGGZvg4OWLZj3VAb7EH+fzRL+Izvh9Bgt74TBeD0AgOQCQV9r/9s815Fj87SvFpgVt9kIt8UKowovVlp1PsZsCmUy4icXviDPlHj7XYBnztuz9ijCbyiMzUXmFhtZH5d4/By0vA6HIWyK6cgicTTcJCzoG0tYk+/D9siCEKYakB51US1MBL6x4X4WyisNvjb2TIfWhyvp3nTjVZ4CLyMc3KvlBrjpuVffvpAWjAYG0V5FgpoMz6fDLyubOa/AAlDpIz+xO5t9mNp93TRCt7h5h6K2osVNTeteVCOkvkOI3wQbenmZfYTwUpzVx3W9yKhN1xIXFs2M3Ao6Zn1voKVnJpw8urZX4OEsfv5UNAgvAtRWq3SehX6l1H+CQEXK0B0M3XprbcKNJRzKMnEWPzC80XC5azf9jIdAHlG7iFrPxAN+H2fhmRCZx0=
2021-06-23T09:22:20.172 INFO:teuthology.kill: gibba022.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEWTwTAXLYeBTGul4N1FS9V6bhl25Qz9xZdwijYI+ctyEUXM380kq2kUFS2uKnMHGupQUIMT/0+l2T93kEpCJfvWjjcALlUXlprM8pkLTGW+ief7Hj7JfU8wwvwurUAX6hRsColgVdqQWdfpc6HINblBklHgTOPUk6hDBuxjrJSuSislxqwJfvWfYustI7zeaTCnljlkjCMQ7ueXf9agPQbGws/RnQr+ffkFGloKrHOAQOA8O+KEGTwFghc/jSKHnOLw/anprnFbsI1l4QkgXUwyXEYwe1xOo6hm/e0d4lYaQe0/Vm6ARll+hC1wzeuzZ7gi3ZOMlUF/+fKvXX9tX57F2tQhPioVj+pQpukbQL8EOYgh8ujM4BB5CFK+Nun0mJSgZvemNv7gpJH97/QRMHIhgX9Udku4Kxmm5jb3w4LfNfxIJ4OAhulwzO/ce+CeNhMXUcmtwPjjDF+r2xbKWqMxmkl1+z3tOCElHsnDq8YPvZK0utEiK+OIeW2cw6erM=
2021-06-23T09:22:20.191 INFO:teuthology.kill:2021-06-23 09:22:20,191.191 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-23T09:22:20.240 INFO:teuthology.kill:2021-06-23 09:22:20,239.239 INFO:teuthology.orchestra.console:Power off gibba022
2021-06-23T09:22:20.385 INFO:teuthology.kill:2021-06-23 09:22:20,385.385 INFO:teuthology.orchestra.console:Power off for gibba022 completed
2021-06-23T09:22:20.522 INFO:teuthology.kill:2021-06-23 09:22:20,521.521 INFO:teuthology.lock.ops:unlocked gibba022.front.sepia.ceph.com
2021-06-23T09:22:21.199 INFO:teuthology.kill:2021-06-23 09:22:21,199.199 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-06-23T09:22:21.248 INFO:teuthology.kill:2021-06-23 09:22:21,247.247 INFO:teuthology.orchestra.console:Power off gibba013
2021-06-23T09:22:21.394 INFO:teuthology.kill:2021-06-23 09:22:21,394.394 INFO:teuthology.orchestra.console:Power off for gibba013 completed
2021-06-23T09:22:21.530 INFO:teuthology.kill:2021-06-23 09:22:21,530.530 INFO:teuthology.lock.ops:unlocked gibba013.front.sepia.ceph.com
2021-06-23T09:24:21.795 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15
2021-06-23T09:24:21.859 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2021-06-23T09:24:21.860 INFO:teuthology.nuke:Checking targets against current locks
2021-06-23T09:24:21.922 INFO:teuthology.nuke:Not nuking gibba013.front.sepia.ceph.com because description doesn't match
2021-06-23T09:24:21.922 INFO:teuthology.nuke:Not nuking gibba022.front.sepia.ceph.com because description doesn't match