2021-06-09T21:18:44.227 INFO:root:teuthology version: 1.1.0-d92ec140 2021-06-09T21:18:44.364 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba025.front.sepia.ceph.com' reimaging 2021-06-09T21:18:44.366 INFO:teuthology.lock.ops:Updating [ubuntu@gibba025.front.sepia.ceph.com]: reset os type and version on server 2021-06-09T21:18:44.366 INFO:teuthology.lock.ops:Updating gibba025.front.sepia.ceph.com on lock server 2021-06-09T21:18:44.383 INFO:teuthology.lock.ops:Node 'ubuntu@gibba025.front.sepia.ceph.com' reimaging is complete 2021-06-09T21:18:44.383 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba026.front.sepia.ceph.com' reimaging 2021-06-09T21:18:44.384 INFO:teuthology.lock.ops:Updating [ubuntu@gibba026.front.sepia.ceph.com]: reset os type and version on server 2021-06-09T21:18:44.384 INFO:teuthology.lock.ops:Updating gibba026.front.sepia.ceph.com on lock server 2021-06-09T21:18:44.399 INFO:teuthology.lock.ops:Node 'ubuntu@gibba026.front.sepia.ceph.com' reimaging is complete 2021-06-09T21:18:44.595 INFO:teuthology.provision.fog.gibba025:Scheduling deploy of ubuntu 20.04 2021-06-09T21:18:44.601 INFO:teuthology.provision.fog.gibba026:Scheduling deploy of ubuntu 20.04 2021-06-09T21:18:44.944 INFO:teuthology.orchestra.console:Power off gibba025 2021-06-09T21:18:44.984 INFO:teuthology.orchestra.console:Power off gibba026 2021-06-09T21:18:45.125 INFO:teuthology.orchestra.console:Power off for gibba025 completed 2021-06-09T21:18:45.141 INFO:teuthology.orchestra.console:Power off for gibba026 completed 2021-06-09T21:18:45.228 INFO:teuthology.orchestra.console:Power on gibba025 2021-06-09T21:18:45.243 INFO:teuthology.orchestra.console:Power on gibba026 2021-06-09T21:18:49.508 INFO:teuthology.orchestra.console:Power on for gibba026 completed 2021-06-09T21:18:49.611 INFO:teuthology.provision.fog.gibba026:Waiting for deploy to finish 2021-06-09T21:18:53.608 INFO:teuthology.orchestra.console:Power on for gibba025 completed 2021-06-09T21:18:53.710 INFO:teuthology.provision.fog.gibba025:Waiting for deploy to finish 2021-06-09T21:23:33.815 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-09T21:23:34.178 INFO:teuthology.provision.fog.gibba025:Node is ready 2021-06-09T21:23:34.183 INFO:teuthology.orchestra.run.gibba025.stdout:gibba025.front.sepia.ceph.com 2021-06-09T21:23:34.189 INFO:teuthology.orchestra.run.gibba025.stdout:172.21.2.125 gibba025.front.sepia.ceph.com gibba025 2021-06-09T21:23:34.411 INFO:teuthology.provision.fog.gibba025:Deploy complete! 2021-06-09T21:23:34.814 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-09T21:23:35.271 INFO:teuthology.provision.fog.gibba026:Node is ready 2021-06-09T21:23:35.280 INFO:teuthology.orchestra.run.gibba026.stdout:gibba026.front.sepia.ceph.com 2021-06-09T21:23:35.284 INFO:teuthology.orchestra.run.gibba026.stdout:172.21.2.126 gibba026.front.sepia.ceph.com gibba026 2021-06-09T21:23:35.598 INFO:teuthology.provision.fog.gibba026:Deploy complete! 2021-06-09T21:23:35.796 INFO:teuthology.lock.ops:Checking gibba025.front.sepia.ceph.com 2021-06-09T21:23:35.797 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-09T21:23:35.820 INFO:teuthology.lock.ops:Checking gibba026.front.sepia.ceph.com 2021-06-09T21:23:35.820 INFO:teuthology.lock.ops:New key found. Updating... 2021-06-09T21:23:35.837 INFO:teuthology.lock.ops:Updating [gibba025.front.sepia.ceph.com]: set os type and version on server 2021-06-09T21:23:35.838 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-09T21:23:36.014 INFO:teuthology.orchestra.run.gibba025.stdout:x86_64 2021-06-09T21:23:36.019 INFO:teuthology.orchestra.run.gibba025.stdout:NAME="Ubuntu" 2021-06-09T21:23:36.019 INFO:teuthology.orchestra.run.gibba025.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-09T21:23:36.019 INFO:teuthology.orchestra.run.gibba025.stdout:ID=ubuntu 2021-06-09T21:23:36.019 INFO:teuthology.orchestra.run.gibba025.stdout:ID_LIKE=debian 2021-06-09T21:23:36.020 INFO:teuthology.orchestra.run.gibba025.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-09T21:23:36.020 INFO:teuthology.orchestra.run.gibba025.stdout:VERSION_ID="20.04" 2021-06-09T21:23:36.020 INFO:teuthology.orchestra.run.gibba025.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-09T21:23:36.020 INFO:teuthology.orchestra.run.gibba025.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-09T21:23:36.021 INFO:teuthology.orchestra.run.gibba025.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-09T21:23:36.021 INFO:teuthology.orchestra.run.gibba025.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-09T21:23:36.021 INFO:teuthology.orchestra.run.gibba025.stdout:VERSION_CODENAME=focal 2021-06-09T21:23:36.021 INFO:teuthology.orchestra.run.gibba025.stdout:UBUNTU_CODENAME=focal 2021-06-09T21:23:36.023 INFO:teuthology.lock.ops:Updating gibba025.front.sepia.ceph.com on lock server 2021-06-09T21:23:36.038 INFO:teuthology.lock.ops:Updating [gibba026.front.sepia.ceph.com]: set os type and version on server 2021-06-09T21:23:36.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-09T21:23:36.211 INFO:teuthology.orchestra.run.gibba026.stdout:x86_64 2021-06-09T21:23:36.221 INFO:teuthology.orchestra.run.gibba026.stdout:NAME="Ubuntu" 2021-06-09T21:23:36.221 INFO:teuthology.orchestra.run.gibba026.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-06-09T21:23:36.222 INFO:teuthology.orchestra.run.gibba026.stdout:ID=ubuntu 2021-06-09T21:23:36.222 INFO:teuthology.orchestra.run.gibba026.stdout:ID_LIKE=debian 2021-06-09T21:23:36.222 INFO:teuthology.orchestra.run.gibba026.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-06-09T21:23:36.222 INFO:teuthology.orchestra.run.gibba026.stdout:VERSION_ID="20.04" 2021-06-09T21:23:36.223 INFO:teuthology.orchestra.run.gibba026.stdout:HOME_URL="https://www.ubuntu.com/" 2021-06-09T21:23:36.223 INFO:teuthology.orchestra.run.gibba026.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-06-09T21:23:36.223 INFO:teuthology.orchestra.run.gibba026.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-06-09T21:23:36.223 INFO:teuthology.orchestra.run.gibba026.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-06-09T21:23:36.223 INFO:teuthology.orchestra.run.gibba026.stdout:VERSION_CODENAME=focal 2021-06-09T21:23:36.224 INFO:teuthology.orchestra.run.gibba026.stdout:UBUNTU_CODENAME=focal 2021-06-09T21:23:36.224 INFO:teuthology.lock.ops:Updating gibba026.front.sepia.ceph.com on lock server 2021-06-09T21:23:36.320 INFO:teuthology.dispatcher.supervisor:Running job 6130026 2021-06-09T21:23:36.321 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-05-23_07:01:03-rados-master-distro-basic-gibba/6130026 --name teuthology-2021-05-23_07:01:03-rados-master-distro-basic-gibba --description rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-start-upgrade 4-wait mon_election/classic} -- /home/teuthworker/archive/teuthology-2021-05-23_07:01:03-rados-master-distro-basic-gibba/6130026/orig.config.yaml 2021-06-09T21:23:36.365 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2021-05-23_07:01:03-rados-master-distro-basic-gibba/6130026 2021-06-09T21:23:36.366 INFO:teuthology.dispatcher.supervisor:Job PID: 12630 2021-06-09T21:23:36.366 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-06-10T09:24:24.163 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-06-10T09:24:24.412 INFO:teuthology.kill:Killing Pids: {12630} 2021-06-10T09:24:25.529 INFO:teuthology.kill:Nuking machines: ['gibba025', 'gibba026'] 2021-06-10T09:24:26.452 INFO:teuthology.kill:2021-06-10 09:24:26,452.452 INFO:teuthology.nuke:targets: 2021-06-10T09:24:26.453 INFO:teuthology.kill: gibba025.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC41r515ysN393JrWCGHCHMomJlcAe8E1vDSzEJt4gqj+lPpejxfM057ze5xop6HKNFSSf2WKDrwmdy3/iuysiefa8/yklg4UBcjQLkE05lIO5JNfEhuWCKxHbNZMXmDEBQn8mHO1ltGyKWqDdbOpA8dnbf6lXm/XY9Pw2e/utW8jYEwnCJqsE1q8/zVxT38Rbcz/BUv+0lDd+toEBQlLJw13jfiwdn2pJnZKdmHjr8iIU0anuY+hKoF9VNm0MBxxgUUTF/aYDJtat67yhpz4GpD+Y90ANhuDUL0xOW95/vqsSZG+PXt09N1xtKJspsVEWqUny2tvu32IVMTjKoHPWANZSziZTIy3FVLAO7Z+73jIuyTJziOQKoN537j5Q076iN+78Ko3mx70Bh4o8qSeL9vRZZCpOJ9G5ld/zh7lenrN33gvnZOzVklFupOzatTLtHVX9nuo7R7sNrQm8nqb7ZXOk41COyZOB6ZNCNx1kfzL4aq8PzETPKNpU14jS6TAk= 2021-06-10T09:24:26.454 INFO:teuthology.kill: gibba026.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzwCVy5cCuCWeKjcHtcisNn/Zhh4LtzvLFsB3g/USOsPa//ea+wt1+UnTCvcJMNtwc7eKRVV72rxbUF46QWBWwkcawMBMZ2EApT/wmdaHqSdwXnMOaoAUaJjdmkV01M4lwF6LoZFIqG3gA0WWAr0ftAOQmeedYYir9cmSEMudiOpukTIWqTul7S/jexgpKtSvLSIPBgHgkOPizUiE1icagTMJKYIfcoazeEiJq0x9PS50Vi7N1t8Hi4Q65bTT5Sb75EXUgKrKJCMrtsejY74Dxy7tl6+a24ExfU41dWiag+VMaaXn+HPj95YEK6U/e9SFO5XwIWn1pnJP+oC1SoR1S7JUxGBmOHOOaRSdiG5XsWQ8uB8RwRsNn3lnjIqC4/MQJph7dM1hIll0sbfA4Ty8jDsm3Rxpa7MExrfhVZtt2KKsQVTXWlzwGDDa71K7eTVgUVY8YCd5Koy8HVYoHeBvcGbNEW/8MQ7H2tAWd8UZ0abFHC2fGKo6QdS7e2kNVLvk= 2021-06-10T09:24:26.454 INFO:teuthology.kill:2021-06-10 09:24:26,452.452 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-10T09:24:26.455 INFO:teuthology.kill:2021-06-10 09:24:26,455.455 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-10T09:24:26.514 INFO:teuthology.kill:2021-06-10 09:24:26,514.514 INFO:teuthology.orchestra.console:Power off gibba026 2021-06-10T09:24:26.527 INFO:teuthology.kill:2021-06-10 09:24:26,527.527 INFO:teuthology.orchestra.console:Power off gibba025 2021-06-10T09:24:34.898 INFO:teuthology.kill:2021-06-10 09:24:34,898.898 INFO:teuthology.orchestra.console:Power off for gibba026 completed 2021-06-10T09:24:39.016 INFO:teuthology.kill:2021-06-10 09:24:39,016.016 INFO:teuthology.orchestra.console:Power off for gibba025 completed 2021-06-10T09:24:39.255 INFO:teuthology.task.internal:roles: ubuntu@gibba025.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2021-06-10T09:24:39.256 INFO:teuthology.task.internal:roles: ubuntu@gibba026.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-06-10T09:24:39.256 INFO:teuthology.misc:Compressing logs... 2021-06-10T09:24:39.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-06-10T09:25:39.309 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d92ec1407c30d60262a45a2844110a5c94d567cc/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-06-10T09:25:39.357 INFO:teuthology.kill:No teuthology processes running 2021-06-10T09:25:40.416 INFO:teuthology.kill:Nuking machines: ['gibba025', 'gibba026'] 2021-06-10T09:25:41.337 INFO:teuthology.kill:2021-06-10 09:25:41,336.336 INFO:teuthology.nuke:targets: 2021-06-10T09:25:41.338 INFO:teuthology.kill: gibba025.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC41r515ysN393JrWCGHCHMomJlcAe8E1vDSzEJt4gqj+lPpejxfM057ze5xop6HKNFSSf2WKDrwmdy3/iuysiefa8/yklg4UBcjQLkE05lIO5JNfEhuWCKxHbNZMXmDEBQn8mHO1ltGyKWqDdbOpA8dnbf6lXm/XY9Pw2e/utW8jYEwnCJqsE1q8/zVxT38Rbcz/BUv+0lDd+toEBQlLJw13jfiwdn2pJnZKdmHjr8iIU0anuY+hKoF9VNm0MBxxgUUTF/aYDJtat67yhpz4GpD+Y90ANhuDUL0xOW95/vqsSZG+PXt09N1xtKJspsVEWqUny2tvu32IVMTjKoHPWANZSziZTIy3FVLAO7Z+73jIuyTJziOQKoN537j5Q076iN+78Ko3mx70Bh4o8qSeL9vRZZCpOJ9G5ld/zh7lenrN33gvnZOzVklFupOzatTLtHVX9nuo7R7sNrQm8nqb7ZXOk41COyZOB6ZNCNx1kfzL4aq8PzETPKNpU14jS6TAk= 2021-06-10T09:25:41.338 INFO:teuthology.kill: gibba026.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzwCVy5cCuCWeKjcHtcisNn/Zhh4LtzvLFsB3g/USOsPa//ea+wt1+UnTCvcJMNtwc7eKRVV72rxbUF46QWBWwkcawMBMZ2EApT/wmdaHqSdwXnMOaoAUaJjdmkV01M4lwF6LoZFIqG3gA0WWAr0ftAOQmeedYYir9cmSEMudiOpukTIWqTul7S/jexgpKtSvLSIPBgHgkOPizUiE1icagTMJKYIfcoazeEiJq0x9PS50Vi7N1t8Hi4Q65bTT5Sb75EXUgKrKJCMrtsejY74Dxy7tl6+a24ExfU41dWiag+VMaaXn+HPj95YEK6U/e9SFO5XwIWn1pnJP+oC1SoR1S7JUxGBmOHOOaRSdiG5XsWQ8uB8RwRsNn3lnjIqC4/MQJph7dM1hIll0sbfA4Ty8jDsm3Rxpa7MExrfhVZtt2KKsQVTXWlzwGDDa71K7eTVgUVY8YCd5Koy8HVYoHeBvcGbNEW/8MQ7H2tAWd8UZ0abFHC2fGKo6QdS7e2kNVLvk= 2021-06-10T09:25:41.357 INFO:teuthology.kill:2021-06-10 09:25:41,356.356 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-10T09:25:41.360 INFO:teuthology.kill:2021-06-10 09:25:41,359.359 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-10T09:25:41.417 INFO:teuthology.kill:2021-06-10 09:25:41,416.416 INFO:teuthology.orchestra.console:Power off gibba026 2021-06-10T09:25:41.432 INFO:teuthology.kill:2021-06-10 09:25:41,432.432 INFO:teuthology.orchestra.console:Power off gibba025 2021-06-10T09:25:41.568 INFO:teuthology.kill:2021-06-10 09:25:41,568.568 INFO:teuthology.orchestra.console:Power off for gibba026 completed 2021-06-10T09:25:41.582 INFO:teuthology.kill:2021-06-10 09:25:41,582.582 INFO:teuthology.orchestra.console:Power off for gibba025 completed 2021-06-10T09:25:41.720 INFO:teuthology.kill:2021-06-10 09:25:41,719.719 INFO:teuthology.lock.ops:unlocked gibba026.front.sepia.ceph.com 2021-06-10T09:25:41.725 INFO:teuthology.kill:2021-06-10 09:25:41,725.725 INFO:teuthology.lock.ops:unlocked gibba025.front.sepia.ceph.com 2021-06-10T09:27:42.031 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15