2022-02-12T17:41:01.824 INFO:root:teuthology version: 1.1.0-7cebb3f2 2022-02-12T17:41:01.975 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging 2022-02-12T17:41:01.976 INFO:teuthology.lock.ops:Updating [ubuntu@smithi001.front.sepia.ceph.com]: reset os type and version on server 2022-02-12T17:41:01.977 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2022-02-12T17:41:01.997 INFO:teuthology.lock.ops:Node 'ubuntu@smithi001.front.sepia.ceph.com' reimaging is complete 2022-02-12T17:41:01.998 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi006.front.sepia.ceph.com' reimaging 2022-02-12T17:41:01.998 INFO:teuthology.lock.ops:Updating [ubuntu@smithi006.front.sepia.ceph.com]: reset os type and version on server 2022-02-12T17:41:01.998 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2022-02-12T17:41:02.023 INFO:teuthology.lock.ops:Node 'ubuntu@smithi006.front.sepia.ceph.com' reimaging is complete 2022-02-12T17:41:02.024 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging 2022-02-12T17:41:02.024 INFO:teuthology.lock.ops:Updating [ubuntu@smithi029.front.sepia.ceph.com]: reset os type and version on server 2022-02-12T17:41:02.024 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2022-02-12T17:41:02.042 INFO:teuthology.lock.ops:Node 'ubuntu@smithi029.front.sepia.ceph.com' reimaging is complete 2022-02-12T17:41:02.244 INFO:teuthology.provision.fog.smithi001:Scheduling deploy of ubuntu 20.04 2022-02-12T17:41:02.260 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of ubuntu 20.04 2022-02-12T17:41:02.268 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of ubuntu 20.04 2022-02-12T17:41:02.602 INFO:teuthology.orchestra.console:Power off smithi001 2022-02-12T17:41:02.641 INFO:teuthology.orchestra.console:Power off smithi029 2022-02-12T17:41:02.657 INFO:teuthology.orchestra.console:Power off smithi006 2022-02-12T17:41:11.006 INFO:teuthology.orchestra.console:Power off for smithi029 completed 2022-02-12T17:41:11.020 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2022-02-12T17:41:11.108 INFO:teuthology.orchestra.console:Power on smithi029 2022-02-12T17:41:11.121 INFO:teuthology.orchestra.console:Power on smithi006 2022-02-12T17:41:15.101 INFO:teuthology.orchestra.console:Power off for smithi001 completed 2022-02-12T17:41:15.202 INFO:teuthology.orchestra.console:Power on smithi001 2022-02-12T17:41:15.370 INFO:teuthology.orchestra.console:Power on for smithi006 completed 2022-02-12T17:41:15.471 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish 2022-02-12T17:41:19.467 INFO:teuthology.orchestra.console:Power on for smithi029 completed 2022-02-12T17:41:19.568 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish 2022-02-12T17:41:23.567 INFO:teuthology.orchestra.console:Power on for smithi001 completed 2022-02-12T17:41:23.668 INFO:teuthology.provision.fog.smithi001:Waiting for deploy to finish 2022-02-12T17:45:56.181 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-12T17:45:56.731 INFO:teuthology.provision.fog.smithi006:Node is ready 2022-02-12T17:45:56.738 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com 2022-02-12T17:45:56.744 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006 2022-02-12T17:45:56.927 INFO:teuthology.provision.fog.smithi006:Deploy complete! 2022-02-12T17:46:03.419 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-12T17:46:03.931 INFO:teuthology.provision.fog.smithi001:Node is ready 2022-02-12T17:46:03.938 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001.front.sepia.ceph.com 2022-02-12T17:46:03.945 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1 smithi001.front.sepia.ceph.com smithi001 2022-02-12T17:46:04.140 INFO:teuthology.provision.fog.smithi001:Deploy complete! 2022-02-12T17:46:51.166 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-12T17:46:51.857 INFO:teuthology.provision.fog.smithi029:Node is ready 2022-02-12T17:46:51.864 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com 2022-02-12T17:46:51.871 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029 2022-02-12T17:46:52.097 INFO:teuthology.provision.fog.smithi029:Deploy complete! 2022-02-12T17:46:52.324 INFO:teuthology.lock.ops:Checking smithi001.front.sepia.ceph.com 2022-02-12T17:46:52.325 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-12T17:46:52.343 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com 2022-02-12T17:46:52.344 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-12T17:46:52.359 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com 2022-02-12T17:46:52.359 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-12T17:46:52.377 INFO:teuthology.lock.ops:Updating [smithi001.front.sepia.ceph.com]: set os type and version on server 2022-02-12T17:46:52.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2022-02-12T17:46:52.643 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2022-02-12T17:46:52.648 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="Ubuntu" 2022-02-12T17:46:52.649 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-12T17:46:52.649 INFO:teuthology.orchestra.run.smithi001.stdout:ID=ubuntu 2022-02-12T17:46:52.649 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE=debian 2022-02-12T17:46:52.649 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-12T17:46:52.650 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="20.04" 2022-02-12T17:46:52.650 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-12T17:46:52.650 INFO:teuthology.orchestra.run.smithi001.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-12T17:46:52.650 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-12T17:46:52.651 INFO:teuthology.orchestra.run.smithi001.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-12T17:46:52.651 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_CODENAME=focal 2022-02-12T17:46:52.651 INFO:teuthology.orchestra.run.smithi001.stdout:UBUNTU_CODENAME=focal 2022-02-12T17:46:52.652 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2022-02-12T17:46:52.671 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server 2022-02-12T17:46:52.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2022-02-12T17:46:52.942 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2022-02-12T17:46:52.992 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Ubuntu" 2022-02-12T17:46:52.993 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-12T17:46:52.993 INFO:teuthology.orchestra.run.smithi006.stdout:ID=ubuntu 2022-02-12T17:46:52.993 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE=debian 2022-02-12T17:46:52.993 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-12T17:46:52.994 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="20.04" 2022-02-12T17:46:52.994 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-12T17:46:52.994 INFO:teuthology.orchestra.run.smithi006.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-12T17:46:52.994 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-12T17:46:52.994 INFO:teuthology.orchestra.run.smithi006.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-12T17:46:52.995 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_CODENAME=focal 2022-02-12T17:46:52.995 INFO:teuthology.orchestra.run.smithi006.stdout:UBUNTU_CODENAME=focal 2022-02-12T17:46:52.995 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2022-02-12T17:46:53.013 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server 2022-02-12T17:46:53.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2022-02-12T18:47:53.322 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/dispatcher/supervisor.py", line 206, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/lock/ops.py", line 317, in reimage_machines update_nodes(reimaged) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/lock/ops.py", line 38, in update_nodes inventory_info = remote.inventory_info File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/remote.py", line 663, in inventory_info node['arch'] = self.arch File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/remote.py", line 320, in arch self._arch = self.sh('uname -m').strip() File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1010, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel. 2022-02-12T18:47:53.427 INFO:teuthology.nuke:Checking targets against current locks 2022-02-12T18:47:53.591 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-12T18:47:53.596 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-12T18:47:53.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-12T18:47:53.679 INFO:teuthology.orchestra.console:Power off smithi001 2022-02-12T18:47:53.696 INFO:teuthology.orchestra.console:Power off smithi006 2022-02-12T18:47:53.716 INFO:teuthology.orchestra.console:Power off smithi029 2022-02-12T18:48:02.079 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2022-02-12T18:48:02.086 INFO:teuthology.orchestra.console:Power off for smithi001 completed 2022-02-12T18:48:02.096 INFO:teuthology.orchestra.console:Power off for smithi029 completed 2022-02-12T18:48:02.238 INFO:teuthology.lock.ops:unlocked: smithi006.front.sepia.ceph.com 2022-02-12T18:48:02.242 INFO:teuthology.lock.ops:unlocked: smithi001.front.sepia.ceph.com 2022-02-12T18:48:02.250 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com 2022-02-12T18:48:02.563 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/dispatcher/supervisor.py", line 52, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/dispatcher/supervisor.py", line 206, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/lock/ops.py", line 317, in reimage_machines update_nodes(reimaged) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/lock/ops.py", line 38, in update_nodes inventory_info = remote.inventory_info File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/remote.py", line 663, in inventory_info node['arch'] = self.arch File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/remote.py", line 320, in arch self._arch = self.sh('uname -m').strip() File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_7cebb3f2319fd6c8340c0f7cd15a137e747fd32e/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1010, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel.