2022-02-06T11:59:27.582 INFO:root:teuthology version: 1.1.0-3094160c 2022-02-06T11:59:27.757 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging 2022-02-06T11:59:27.760 INFO:teuthology.lock.ops:Updating [ubuntu@smithi068.front.sepia.ceph.com]: reset os type and version on server 2022-02-06T11:59:27.761 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server 2022-02-06T11:59:27.781 INFO:teuthology.lock.ops:Node 'ubuntu@smithi068.front.sepia.ceph.com' reimaging is complete 2022-02-06T11:59:27.781 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi072.front.sepia.ceph.com' reimaging 2022-02-06T11:59:27.782 INFO:teuthology.lock.ops:Updating [ubuntu@smithi072.front.sepia.ceph.com]: reset os type and version on server 2022-02-06T11:59:27.782 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2022-02-06T11:59:27.800 INFO:teuthology.lock.ops:Node 'ubuntu@smithi072.front.sepia.ceph.com' reimaging is complete 2022-02-06T11:59:27.800 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging 2022-02-06T11:59:27.801 INFO:teuthology.lock.ops:Updating [ubuntu@smithi114.front.sepia.ceph.com]: reset os type and version on server 2022-02-06T11:59:27.801 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2022-02-06T11:59:27.819 INFO:teuthology.lock.ops:Node 'ubuntu@smithi114.front.sepia.ceph.com' reimaging is complete 2022-02-06T11:59:28.079 INFO:teuthology.provision.fog.smithi068:Scheduling deploy of ubuntu 20.04 2022-02-06T11:59:28.085 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of ubuntu 20.04 2022-02-06T11:59:28.113 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of ubuntu 20.04 2022-02-06T11:59:28.561 INFO:teuthology.orchestra.console:Power off smithi114 2022-02-06T11:59:28.579 INFO:teuthology.orchestra.console:Power off smithi068 2022-02-06T11:59:28.603 INFO:teuthology.orchestra.console:Power off smithi072 2022-02-06T11:59:28.715 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2022-02-06T11:59:28.737 INFO:teuthology.orchestra.console:Power off for smithi068 completed 2022-02-06T11:59:28.749 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2022-02-06T11:59:28.817 INFO:teuthology.orchestra.console:Power on smithi114 2022-02-06T11:59:28.844 INFO:teuthology.orchestra.console:Power on smithi068 2022-02-06T11:59:28.851 INFO:teuthology.orchestra.console:Power on smithi072 2022-02-06T11:59:37.190 INFO:teuthology.orchestra.console:Power on for smithi114 completed 2022-02-06T11:59:37.210 INFO:teuthology.orchestra.console:Power on for smithi068 completed 2022-02-06T11:59:37.222 INFO:teuthology.orchestra.console:Power on for smithi072 completed 2022-02-06T11:59:37.292 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish 2022-02-06T11:59:37.312 INFO:teuthology.provision.fog.smithi068:Waiting for deploy to finish 2022-02-06T11:59:37.324 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2022-02-06T12:04:34.880 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-06T12:04:35.373 INFO:teuthology.provision.fog.smithi114:Node is ready 2022-02-06T12:04:35.421 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com 2022-02-06T12:04:35.428 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114 2022-02-06T12:04:35.558 INFO:teuthology.provision.fog.smithi114:Deploy complete! 2022-02-06T12:04:50.078 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-06T12:04:50.549 INFO:teuthology.provision.fog.smithi068:Node is ready 2022-02-06T12:04:50.556 INFO:teuthology.orchestra.run.smithi068.stdout:smithi068.front.sepia.ceph.com 2022-02-06T12:04:50.563 INFO:teuthology.orchestra.run.smithi068.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068 2022-02-06T12:04:50.795 INFO:teuthology.provision.fog.smithi068:Deploy complete! 2022-02-06T12:05:45.602 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-06T12:05:46.293 INFO:teuthology.provision.fog.smithi072:Node is ready 2022-02-06T12:05:46.300 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072.front.sepia.ceph.com 2022-02-06T12:05:46.308 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72 smithi072.front.sepia.ceph.com smithi072 2022-02-06T12:05:46.501 INFO:teuthology.provision.fog.smithi072:Deploy complete! 2022-02-06T12:05:46.752 INFO:teuthology.lock.ops:Checking smithi068.front.sepia.ceph.com 2022-02-06T12:05:46.753 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-06T12:05:46.779 INFO:teuthology.lock.ops:Checking smithi072.front.sepia.ceph.com 2022-02-06T12:05:46.779 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-06T12:05:46.798 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com 2022-02-06T12:05:46.799 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-06T12:05:46.821 INFO:teuthology.lock.ops:Updating [smithi068.front.sepia.ceph.com]: set os type and version on server 2022-02-06T12:05:46.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi068.front.sepia.ceph.com' 2022-02-06T12:05:47.078 INFO:teuthology.orchestra.run.smithi068.stdout:x86_64 2022-02-06T12:05:47.085 INFO:teuthology.orchestra.run.smithi068.stdout:NAME="Ubuntu" 2022-02-06T12:05:47.085 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-06T12:05:47.086 INFO:teuthology.orchestra.run.smithi068.stdout:ID=ubuntu 2022-02-06T12:05:47.086 INFO:teuthology.orchestra.run.smithi068.stdout:ID_LIKE=debian 2022-02-06T12:05:47.086 INFO:teuthology.orchestra.run.smithi068.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-06T12:05:47.086 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_ID="20.04" 2022-02-06T12:05:47.087 INFO:teuthology.orchestra.run.smithi068.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-06T12:05:47.087 INFO:teuthology.orchestra.run.smithi068.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-06T12:05:47.087 INFO:teuthology.orchestra.run.smithi068.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-06T12:05:47.087 INFO:teuthology.orchestra.run.smithi068.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-06T12:05:47.088 INFO:teuthology.orchestra.run.smithi068.stdout:VERSION_CODENAME=focal 2022-02-06T12:05:47.088 INFO:teuthology.orchestra.run.smithi068.stdout:UBUNTU_CODENAME=focal 2022-02-06T12:05:47.089 INFO:teuthology.lock.ops:Updating smithi068.front.sepia.ceph.com on lock server 2022-02-06T12:05:47.109 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: set os type and version on server 2022-02-06T12:05:47.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2022-02-06T12:05:47.334 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2022-02-06T12:05:47.340 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Ubuntu" 2022-02-06T12:05:47.341 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-06T12:05:47.341 INFO:teuthology.orchestra.run.smithi072.stdout:ID=ubuntu 2022-02-06T12:05:47.341 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE=debian 2022-02-06T12:05:47.341 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-06T12:05:47.341 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="20.04" 2022-02-06T12:05:47.342 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-06T12:05:47.342 INFO:teuthology.orchestra.run.smithi072.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-06T12:05:47.342 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-06T12:05:47.342 INFO:teuthology.orchestra.run.smithi072.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-06T12:05:47.343 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_CODENAME=focal 2022-02-06T12:05:47.343 INFO:teuthology.orchestra.run.smithi072.stdout:UBUNTU_CODENAME=focal 2022-02-06T12:05:47.343 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2022-02-06T12:05:47.362 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server 2022-02-06T12:05:47.363 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2022-02-06T13:06:47.710 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/dispatcher/supervisor.py", line 206, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/lock/ops.py", line 317, in reimage_machines update_nodes(reimaged) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/lock/ops.py", line 38, in update_nodes inventory_info = remote.inventory_info File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/remote.py", line 663, in inventory_info node['arch'] = self.arch File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/remote.py", line 320, in arch self._arch = self.sh('uname -m').strip() File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1010, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel. 2022-02-06T13:06:47.828 INFO:teuthology.nuke:Checking targets against current locks 2022-02-06T13:06:47.963 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-06T13:06:47.967 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-06T13:06:47.969 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-06T13:06:48.053 INFO:teuthology.orchestra.console:Power off smithi068 2022-02-06T13:06:48.068 INFO:teuthology.orchestra.console:Power off smithi072 2022-02-06T13:06:48.080 INFO:teuthology.orchestra.console:Power off smithi114 2022-02-06T13:06:56.451 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2022-02-06T13:06:56.459 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2022-02-06T13:06:56.609 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com 2022-02-06T13:06:56.613 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com 2022-02-06T13:07:00.557 INFO:teuthology.orchestra.console:Power off for smithi068 completed 2022-02-06T13:07:00.706 INFO:teuthology.lock.ops:unlocked: smithi068.front.sepia.ceph.com 2022-02-06T13:07:01.006 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/dispatcher/supervisor.py", line 52, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/dispatcher/supervisor.py", line 206, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/lock/ops.py", line 317, in reimage_machines update_nodes(reimaged) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/lock/ops.py", line 38, in update_nodes inventory_info = remote.inventory_info File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/remote.py", line 663, in inventory_info node['arch'] = self.arch File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/remote.py", line 320, in arch self._arch = self.sh('uname -m').strip() File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_3094160cc590b786a43d55faaf7c99d5de71ce56/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1010, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel.