2021-05-06T04:58:48.979 INFO:root:teuthology version: 1.1.0-19220a3b 2021-05-06T04:58:49.082 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging 2021-05-06T04:58:49.083 INFO:teuthology.lock.ops:Updating [ubuntu@smithi001.front.sepia.ceph.com]: reset os type and version on server 2021-05-06T04:58:49.083 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2021-05-06T04:58:49.099 INFO:teuthology.lock.ops:Node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging is complete 2021-05-06T04:58:49.099 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging 2021-05-06T04:58:49.099 INFO:teuthology.lock.ops:Updating [ubuntu@smithi136.front.sepia.ceph.com]: reset os type and version on server 2021-05-06T04:58:49.100 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-05-06T04:58:49.116 INFO:teuthology.lock.ops:Node 'ubuntu@smithi136.front.sepia.ceph.com' reimaging is complete 2021-05-06T04:58:49.305 INFO:teuthology.provision.fog.smithi001:Scheduling deploy of ubuntu 20.04 2021-05-06T04:58:49.323 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of ubuntu 20.04 2021-05-06T04:58:49.679 INFO:teuthology.orchestra.console:Power off smithi136 2021-05-06T04:58:49.693 INFO:teuthology.orchestra.console:Power off smithi001 2021-05-06T04:58:58.059 INFO:teuthology.orchestra.console:Power off for smithi001 completed 2021-05-06T04:58:58.160 INFO:teuthology.orchestra.console:Power on smithi001 2021-05-06T04:59:02.164 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-05-06T04:59:02.266 INFO:teuthology.orchestra.console:Power on smithi136 2021-05-06T04:59:06.526 INFO:teuthology.orchestra.console:Power on for smithi136 completed 2021-05-06T04:59:06.530 INFO:teuthology.orchestra.console:Power on for smithi001 completed 2021-05-06T04:59:06.628 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2021-05-06T04:59:06.632 INFO:teuthology.provision.fog.smithi001:Waiting for deploy to finish 2021-05-06T05:03:44.493 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-06T05:03:45.019 INFO:teuthology.provision.fog.smithi001:Node is ready 2021-05-06T05:03:45.026 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001.front.sepia.ceph.com 2021-05-06T05:03:45.033 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1 smithi001.front.sepia.ceph.com smithi001 2021-05-06T05:03:45.282 INFO:teuthology.provision.fog.smithi001:Deploy complete! 2021-05-06T05:04:13.159 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-06T05:04:13.662 INFO:teuthology.provision.fog.smithi136:Node is ready 2021-05-06T05:04:13.711 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2021-05-06T05:04:13.718 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2021-05-06T05:04:13.866 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2021-05-06T05:04:14.031 INFO:teuthology.lock.ops:Checking smithi001.front.sepia.ceph.com 2021-05-06T05:04:14.032 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-06T05:04:14.048 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2021-05-06T05:04:14.049 INFO:teuthology.lock.ops:New key found. Updating... 2021-05-06T05:04:14.064 INFO:teuthology.lock.ops:Updating [smithi001.front.sepia.ceph.com]: set os type and version on server 2021-05-06T05:04:14.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-06T05:04:14.327 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2021-05-06T05:04:14.334 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="Ubuntu" 2021-05-06T05:04:14.334 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-05-06T05:04:14.334 INFO:teuthology.orchestra.run.smithi001.stdout:ID=ubuntu 2021-05-06T05:04:14.334 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE=debian 2021-05-06T05:04:14.335 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-05-06T05:04:14.335 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="20.04" 2021-05-06T05:04:14.335 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.ubuntu.com/" 2021-05-06T05:04:14.335 INFO:teuthology.orchestra.run.smithi001.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-05-06T05:04:14.336 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-05-06T05:04:14.336 INFO:teuthology.orchestra.run.smithi001.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-05-06T05:04:14.336 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_CODENAME=focal 2021-05-06T05:04:14.336 INFO:teuthology.orchestra.run.smithi001.stdout:UBUNTU_CODENAME=focal 2021-05-06T05:04:14.338 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2021-05-06T05:04:14.355 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2021-05-06T05:04:14.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-06T05:04:14.522 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2021-05-06T05:04:14.529 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2021-05-06T05:04:14.529 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-05-06T05:04:14.529 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2021-05-06T05:04:14.530 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2021-05-06T05:04:14.530 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-05-06T05:04:14.530 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="20.04" 2021-05-06T05:04:14.530 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2021-05-06T05:04:14.530 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-05-06T05:04:14.531 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-05-06T05:04:14.531 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-05-06T05:04:14.531 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=focal 2021-05-06T05:04:14.531 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=focal 2021-05-06T05:04:14.532 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2021-05-06T05:04:14.615 INFO:teuthology.dispatcher.supervisor:Running job 6098043 2021-05-06T05:04:14.616 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/virtualenv/bin/teuthology -v --owner scheduled_ksirivad@teuthology --archive /home/teuthworker/archive/ksirivad-2021-05-05_13:55:15-rados-wip-pool-start-out-with-1-pg-distro-basic-smithi/6098043 --name ksirivad-2021-05-05_13:55:15-rados-wip-pool-start-out-with-1-pg-distro-basic-smithi --description rados/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait mon_election/classic} -- /home/teuthworker/archive/ksirivad-2021-05-05_13:55:15-rados-wip-pool-start-out-with-1-pg-distro-basic-smithi/6098043/orig.config.yaml 2021-05-06T05:04:14.619 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/ksirivad-2021-05-05_13:55:15-rados-wip-pool-start-out-with-1-pg-distro-basic-smithi/6098043 2021-05-06T05:04:14.620 INFO:teuthology.dispatcher.supervisor:Job PID: 2330 2021-05-06T05:04:14.620 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2021-05-06T17:05:09.189 WARNING:teuthology.dispatcher.supervisor:Job ran longer than 43200s. Killing... 2021-05-06T17:05:09.405 INFO:teuthology.kill:Killing Pids: {2330} 2021-05-06T17:05:10.493 INFO:teuthology.kill:Nuking machines: ['smithi001', 'smithi136'] 2021-05-06T17:05:11.516 INFO:teuthology.kill:2021-05-06 17:05:11,516.516 INFO:teuthology.nuke:targets: 2021-05-06T17:05:11.517 INFO:teuthology.kill: smithi001.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZuA7hD765BZkeRpKLYIdhKx7GQH8QrjwT4ZKwKadCUWxnkkOXbFmpMkjxhr3z1PcPYsUkwhwiOwUIbqtOv1cDc65qZ2fP/EG01elDo0/Q1kFkmppfnrrIE13znNqhCFx7Qf96vRKJu4+KCR/GVjtz1JgCEKVXfpNd7HatsjX8HXVlAaQKFbgKKbdH66AfuvoJIE293vqIfvdc0/6I+ajADje5L2A8SJCwX1wPojNlpITP6dCYwT2c7eumYQyl9NR5XRfYLJLgzG5yTgvIYSJED5JA8iJf2QCISNpTNY6lWEM3+v89mxTy+amfdjjTa/7lvzTQukE8/ub1se9+PKnEYgOnyuvi9ukmO8LQRqv6USGCcMQFxHK4xHAZdfcJr3qtX3HHzR2xtHxile1WEu396P0dw/jyMKhHC/EYd9YOpQAWcSkOnV98wv3mn66uyMFGfvH5PK3EJPCK/53fsMqfemjd3YhxGMeM+5TdV30BjzTJgRF5Q5ipu23+AI6BOw0= 2021-05-06T17:05:11.517 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVoSZHi1CpZ/1nmk1GEqHDtM7FCSY7pzXyvNWbx3SG7gC6WTM65+cHruekTq1cbXMOAL6S0et/ov8DGSii+aKqgZrr3NEWBiS7puu7iQ/qcJBj9G36PuBd3f0k3tb1sQ54xdTU67000JU3mYZ0EpTkl8KxEhOfKd6Y9UGeIuDAi3WFzWWAG5RR+SZfsuva7PpqCP5Ae79XWJ8TP3NMPkiE5Yje8ylNsLnWqVyU9F1R4Z1OHljxII01A8epsHQvqUE8gKysnMd0fBOGROsOSd12EOjYiap5WZ9nYPsIBJrfe6hIVw/TOMSMiBj7PtmL2VOxZIheeHGE4EuUgVMqkwQjPTly5ZSL26ISstBYoJp/9KzMfinmy5Zut+R9x9rvYFGqmRRWy/ccVKSAk1jnEoZ8IMVF72NKkIM1yxi2vCggAzKcWKBM5hnaLSCtzAxYjgHbg7esd7mobgx+told7pqeiKYvutLdO1ZMveqea9/AsK6k0ltsvNfYr+Nc/CCJ3Yk= 2021-05-06T17:05:11.518 INFO:teuthology.kill:2021-05-06 17:05:11,516.516 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-06T17:05:11.518 INFO:teuthology.kill:2021-05-06 17:05:11,518.518 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-06T17:05:11.579 INFO:teuthology.kill:2021-05-06 17:05:11,578.578 INFO:teuthology.orchestra.console:Power off smithi136 2021-05-06T17:05:11.591 INFO:teuthology.kill:2021-05-06 17:05:11,591.591 INFO:teuthology.orchestra.console:Power off smithi001 2021-05-06T17:05:24.057 INFO:teuthology.kill:2021-05-06 17:05:24,056.056 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-05-06T17:05:24.089 INFO:teuthology.kill:2021-05-06 17:05:24,088.088 INFO:teuthology.orchestra.console:Power off for smithi001 completed 2021-05-06T17:05:24.315 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2021-05-06T17:05:24.316 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2021-05-06T17:05:24.316 INFO:teuthology.misc:Compressing logs... 2021-05-06T17:05:24.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-05-06T17:05:58.876 ERROR:teuthology.dispatcher.supervisor:Could not save logs Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 231, in run_with_watchdog teuth_config.archive_base, job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 289, in transfer_archives compress_logs(ctx, log_path) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/misc.py", line 1378, in compress_logs wait=False, File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/cluster.py", line 64, in return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/orchestra/run.py", line 434, in run transport = client.get_transport() AttributeError: 'NoneType' object has no attribute 'get_transport' 2021-05-06T17:05:58.923 INFO:teuthology.kill:No teuthology processes running 2021-05-06T17:05:59.966 INFO:teuthology.kill:Nuking machines: ['smithi001', 'smithi136'] 2021-05-06T17:06:00.883 INFO:teuthology.kill:2021-05-06 17:06:00,883.883 INFO:teuthology.nuke:targets: 2021-05-06T17:06:00.884 INFO:teuthology.kill: smithi001.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZuA7hD765BZkeRpKLYIdhKx7GQH8QrjwT4ZKwKadCUWxnkkOXbFmpMkjxhr3z1PcPYsUkwhwiOwUIbqtOv1cDc65qZ2fP/EG01elDo0/Q1kFkmppfnrrIE13znNqhCFx7Qf96vRKJu4+KCR/GVjtz1JgCEKVXfpNd7HatsjX8HXVlAaQKFbgKKbdH66AfuvoJIE293vqIfvdc0/6I+ajADje5L2A8SJCwX1wPojNlpITP6dCYwT2c7eumYQyl9NR5XRfYLJLgzG5yTgvIYSJED5JA8iJf2QCISNpTNY6lWEM3+v89mxTy+amfdjjTa/7lvzTQukE8/ub1se9+PKnEYgOnyuvi9ukmO8LQRqv6USGCcMQFxHK4xHAZdfcJr3qtX3HHzR2xtHxile1WEu396P0dw/jyMKhHC/EYd9YOpQAWcSkOnV98wv3mn66uyMFGfvH5PK3EJPCK/53fsMqfemjd3YhxGMeM+5TdV30BjzTJgRF5Q5ipu23+AI6BOw0= 2021-05-06T17:06:00.884 INFO:teuthology.kill: smithi136.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCVoSZHi1CpZ/1nmk1GEqHDtM7FCSY7pzXyvNWbx3SG7gC6WTM65+cHruekTq1cbXMOAL6S0et/ov8DGSii+aKqgZrr3NEWBiS7puu7iQ/qcJBj9G36PuBd3f0k3tb1sQ54xdTU67000JU3mYZ0EpTkl8KxEhOfKd6Y9UGeIuDAi3WFzWWAG5RR+SZfsuva7PpqCP5Ae79XWJ8TP3NMPkiE5Yje8ylNsLnWqVyU9F1R4Z1OHljxII01A8epsHQvqUE8gKysnMd0fBOGROsOSd12EOjYiap5WZ9nYPsIBJrfe6hIVw/TOMSMiBj7PtmL2VOxZIheeHGE4EuUgVMqkwQjPTly5ZSL26ISstBYoJp/9KzMfinmy5Zut+R9x9rvYFGqmRRWy/ccVKSAk1jnEoZ8IMVF72NKkIM1yxi2vCggAzKcWKBM5hnaLSCtzAxYjgHbg7esd7mobgx+told7pqeiKYvutLdO1ZMveqea9/AsK6k0ltsvNfYr+Nc/CCJ3Yk= 2021-05-06T17:06:00.905 INFO:teuthology.kill:2021-05-06 17:06:00,905.905 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-06T17:06:00.908 INFO:teuthology.kill:2021-05-06 17:06:00,908.908 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-06T17:06:00.966 INFO:teuthology.kill:2021-05-06 17:06:00,965.965 INFO:teuthology.orchestra.console:Power off smithi136 2021-05-06T17:06:00.979 INFO:teuthology.kill:2021-05-06 17:06:00,978.978 INFO:teuthology.orchestra.console:Power off smithi001 2021-05-06T17:06:01.110 INFO:teuthology.kill:2021-05-06 17:06:01,110.110 INFO:teuthology.orchestra.console:Power off for smithi136 completed 2021-05-06T17:06:01.118 INFO:teuthology.kill:2021-05-06 17:06:01,118.118 INFO:teuthology.orchestra.console:Power off for smithi001 completed 2021-05-06T17:06:01.257 INFO:teuthology.kill:2021-05-06 17:06:01,257.257 INFO:teuthology.lock.ops:unlocked smithi136.front.sepia.ceph.com 2021-05-06T17:06:01.261 INFO:teuthology.kill:2021-05-06 17:06:01,261.261 INFO:teuthology.lock.ops:unlocked smithi001.front.sepia.ceph.com 2021-05-06T17:08:01.549 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2021-05-06T17:08:01.617 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2021-05-06T17:08:01.618 INFO:teuthology.nuke:Checking targets against current locks 2021-05-06T17:08:01.690 INFO:teuthology.nuke:Not nuking smithi001.front.sepia.ceph.com because description doesn't match 2021-05-06T17:08:01.691 INFO:teuthology.nuke:Not nuking smithi136.front.sepia.ceph.com because description doesn't match