2022-04-05T18:01:14.727 INFO:root:teuthology version: 1.1.0-d877b023 2022-04-05T18:01:14.875 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi022.front.sepia.ceph.com' reimaging 2022-04-05T18:01:14.876 INFO:teuthology.lock.ops:Updating [ubuntu@smithi022.front.sepia.ceph.com]: reset os type and version on server 2022-04-05T18:01:14.877 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2022-04-05T18:01:14.894 INFO:teuthology.lock.ops:Node 'ubuntu@smithi022.front.sepia.ceph.com' reimaging is complete 2022-04-05T18:01:14.895 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging 2022-04-05T18:01:14.895 INFO:teuthology.lock.ops:Updating [ubuntu@smithi169.front.sepia.ceph.com]: reset os type and version on server 2022-04-05T18:01:14.895 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2022-04-05T18:01:14.913 INFO:teuthology.lock.ops:Node 'ubuntu@smithi169.front.sepia.ceph.com' reimaging is complete 2022-04-05T18:01:14.913 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging 2022-04-05T18:01:14.913 INFO:teuthology.lock.ops:Updating [ubuntu@smithi174.front.sepia.ceph.com]: reset os type and version on server 2022-04-05T18:01:14.914 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2022-04-05T18:01:14.930 INFO:teuthology.lock.ops:Node 'ubuntu@smithi174.front.sepia.ceph.com' reimaging is complete 2022-04-05T18:01:15.114 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of ubuntu 20.04 2022-04-05T18:01:15.117 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of ubuntu 20.04 2022-04-05T18:01:15.155 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of ubuntu 20.04 2022-04-05T18:01:15.456 INFO:teuthology.orchestra.console:Power off smithi022 2022-04-05T18:01:15.494 INFO:teuthology.orchestra.console:Power off smithi169 2022-04-05T18:01:15.563 INFO:teuthology.orchestra.console:Power off smithi174 2022-04-05T18:01:23.826 INFO:teuthology.orchestra.console:Power off for smithi022 completed 2022-04-05T18:01:23.852 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2022-04-05T18:01:23.928 INFO:teuthology.orchestra.console:Power on smithi022 2022-04-05T18:01:23.954 INFO:teuthology.orchestra.console:Power on smithi169 2022-04-05T18:01:28.033 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2022-04-05T18:01:28.134 INFO:teuthology.orchestra.console:Power on smithi174 2022-04-05T18:01:32.286 INFO:teuthology.orchestra.console:Power on for smithi022 completed 2022-04-05T18:01:32.308 INFO:teuthology.orchestra.console:Power on for smithi169 completed 2022-04-05T18:01:32.383 INFO:teuthology.orchestra.console:Power on for smithi174 completed 2022-04-05T18:01:32.387 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish 2022-04-05T18:01:32.409 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish 2022-04-05T18:01:32.484 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2022-04-05T18:06:10.288 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-05T18:06:10.807 INFO:teuthology.provision.fog.smithi022:Node is ready 2022-04-05T18:06:10.813 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com 2022-04-05T18:06:10.821 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2022-04-05T18:06:10.966 INFO:teuthology.provision.fog.smithi022:Deploy complete! 2022-04-05T18:06:29.838 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-05T18:06:30.381 INFO:teuthology.provision.fog.smithi174:Node is ready 2022-04-05T18:06:30.388 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2022-04-05T18:06:30.439 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2022-04-05T18:06:30.651 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2022-04-05T18:06:34.326 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-04-05T18:06:34.911 INFO:teuthology.provision.fog.smithi169:Node is ready 2022-04-05T18:06:34.918 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com 2022-04-05T18:06:34.967 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2022-04-05T18:06:35.145 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2022-04-05T18:06:35.371 INFO:teuthology.lock.ops:Checking smithi022.front.sepia.ceph.com 2022-04-05T18:06:35.372 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-05T18:06:35.408 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2022-04-05T18:06:35.408 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-05T18:06:35.426 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2022-04-05T18:06:35.426 INFO:teuthology.lock.ops:New key found. Updating... 2022-04-05T18:06:35.444 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: set os type and version on server 2022-04-05T18:06:35.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2022-04-05T18:06:35.710 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2022-04-05T18:06:35.717 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="Ubuntu" 2022-04-05T18:06:35.718 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-04-05T18:06:35.718 INFO:teuthology.orchestra.run.smithi022.stdout:ID=ubuntu 2022-04-05T18:06:35.718 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE=debian 2022-04-05T18:06:35.718 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-04-05T18:06:35.719 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="20.04" 2022-04-05T18:06:35.719 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.ubuntu.com/" 2022-04-05T18:06:35.719 INFO:teuthology.orchestra.run.smithi022.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-04-05T18:06:35.719 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-04-05T18:06:35.720 INFO:teuthology.orchestra.run.smithi022.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-04-05T18:06:35.720 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_CODENAME=focal 2022-04-05T18:06:35.720 INFO:teuthology.orchestra.run.smithi022.stdout:UBUNTU_CODENAME=focal 2022-04-05T18:06:35.721 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2022-04-05T18:06:35.740 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2022-04-05T18:06:35.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2022-04-05T18:06:35.954 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2022-04-05T18:06:35.960 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="Ubuntu" 2022-04-05T18:06:35.961 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2022-04-05T18:06:35.961 INFO:teuthology.orchestra.run.smithi169.stdout:ID=ubuntu 2022-04-05T18:06:35.961 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE=debian 2022-04-05T18:06:35.961 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2022-04-05T18:06:35.961 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="20.04" 2022-04-05T18:06:35.962 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://www.ubuntu.com/" 2022-04-05T18:06:35.962 INFO:teuthology.orchestra.run.smithi169.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-04-05T18:06:35.962 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-04-05T18:06:35.962 INFO:teuthology.orchestra.run.smithi169.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-04-05T18:06:35.963 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_CODENAME=focal 2022-04-05T18:06:35.963 INFO:teuthology.orchestra.run.smithi169.stdout:UBUNTU_CODENAME=focal 2022-04-05T18:06:35.963 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2022-04-05T18:06:35.987 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2022-04-05T18:06:35.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2022-04-05T19:06:36.175 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/dispatcher/supervisor.py", line 207, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/lock/ops.py", line 326, in reimage_machines update_nodes(reimaged) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/lock/ops.py", line 38, in update_nodes inventory_info = remote.inventory_info File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/remote.py", line 663, in inventory_info node['arch'] = self.arch File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/remote.py", line 320, in arch self._arch = self.sh('uname -m').strip() File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1010, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel. 2022-04-05T19:06:36.287 INFO:teuthology.nuke:Checking targets against current locks 2022-04-05T19:06:36.417 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-05T19:06:36.422 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-05T19:06:36.423 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-04-05T19:06:36.501 INFO:teuthology.orchestra.console:Power off smithi169 2022-04-05T19:06:36.516 INFO:teuthology.orchestra.console:Power off smithi174 2022-04-05T19:06:36.528 INFO:teuthology.orchestra.console:Power off smithi022 2022-04-05T19:06:44.903 INFO:teuthology.orchestra.console:Power off for smithi022 completed 2022-04-05T19:06:45.051 INFO:teuthology.lock.ops:unlocked: smithi022.front.sepia.ceph.com 2022-04-05T19:06:48.999 INFO:teuthology.orchestra.console:Power off for smithi169 completed 2022-04-05T19:06:49.005 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2022-04-05T19:06:49.143 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com 2022-04-05T19:06:49.146 INFO:teuthology.lock.ops:unlocked: smithi174.front.sepia.ceph.com 2022-04-05T19:06:49.431 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/dispatcher/supervisor.py", line 52, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/dispatcher/supervisor.py", line 207, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/lock/ops.py", line 326, in reimage_machines update_nodes(reimaged) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/lock/ops.py", line 38, in update_nodes inventory_info = remote.inventory_info File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/remote.py", line 663, in inventory_info node['arch'] = self.arch File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/remote.py", line 320, in arch self._arch = self.sh('uname -m').strip() File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_d877b023166308f282187444de8e8e725ffb27d2/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1010, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel.