2022-02-23T03:29:23.822 INFO:root:teuthology version: 1.1.0-eea25212 2022-02-23T03:29:23.970 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi072.front.sepia.ceph.com' reimaging 2022-02-23T03:29:23.971 INFO:teuthology.lock.ops:Updating [ubuntu@smithi072.front.sepia.ceph.com]: reset os type and version on server 2022-02-23T03:29:23.972 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2022-02-23T03:29:23.997 INFO:teuthology.lock.ops:Node 'ubuntu@smithi072.front.sepia.ceph.com' reimaging is complete 2022-02-23T03:29:23.998 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi159.front.sepia.ceph.com' reimaging 2022-02-23T03:29:23.999 INFO:teuthology.lock.ops:Updating [ubuntu@smithi159.front.sepia.ceph.com]: reset os type and version on server 2022-02-23T03:29:23.999 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-02-23T03:29:24.020 INFO:teuthology.lock.ops:Node 'ubuntu@smithi159.front.sepia.ceph.com' reimaging is complete 2022-02-23T03:29:24.020 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging 2022-02-23T03:29:24.020 INFO:teuthology.lock.ops:Updating [ubuntu@smithi172.front.sepia.ceph.com]: reset os type and version on server 2022-02-23T03:29:24.021 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2022-02-23T03:29:24.038 INFO:teuthology.lock.ops:Node 'ubuntu@smithi172.front.sepia.ceph.com' reimaging is complete 2022-02-23T03:29:24.211 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of ubuntu 20.04 2022-02-23T03:29:24.225 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of ubuntu 20.04 2022-02-23T03:29:24.240 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of ubuntu 20.04 2022-02-23T03:29:24.500 INFO:teuthology.orchestra.console:Power off smithi072 2022-02-23T03:29:24.518 INFO:teuthology.orchestra.console:Power off smithi159 2022-02-23T03:29:24.552 INFO:teuthology.orchestra.console:Power off smithi172 2022-02-23T03:29:32.888 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-02-23T03:29:32.990 INFO:teuthology.orchestra.console:Power on smithi159 2022-02-23T03:29:36.997 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2022-02-23T03:29:37.021 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2022-02-23T03:29:37.099 INFO:teuthology.orchestra.console:Power on smithi072 2022-02-23T03:29:37.126 INFO:teuthology.orchestra.console:Power on smithi172 2022-02-23T03:29:41.352 INFO:teuthology.orchestra.console:Power on for smithi159 completed 2022-02-23T03:29:41.360 INFO:teuthology.orchestra.console:Power on for smithi072 completed 2022-02-23T03:29:41.373 INFO:teuthology.orchestra.console:Power on for smithi172 completed 2022-02-23T03:29:41.453 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2022-02-23T03:29:41.462 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2022-02-23T03:29:41.473 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2022-02-23T03:34:58.831 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-23T03:34:59.341 INFO:teuthology.provision.fog.smithi159:Node is ready 2022-02-23T03:34:59.348 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2022-02-23T03:34:59.356 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2022-02-23T03:34:59.487 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2022-02-23T03:35:02.212 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-23T03:35:02.680 INFO:teuthology.provision.fog.smithi172:Node is ready 2022-02-23T03:35:02.687 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com 2022-02-23T03:35:02.694 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2022-02-23T03:35:02.938 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2022-02-23T03:35:48.645 INFO:teuthology.orchestra.run:Running command with timeout 600 2022-02-23T03:35:49.311 INFO:teuthology.provision.fog.smithi072:Node is ready 2022-02-23T03:35:49.318 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072.front.sepia.ceph.com 2022-02-23T03:35:49.367 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72 smithi072.front.sepia.ceph.com smithi072 2022-02-23T03:35:49.640 INFO:teuthology.provision.fog.smithi072:Deploy complete! 2022-02-23T03:35:49.871 INFO:teuthology.lock.ops:Checking smithi072.front.sepia.ceph.com 2022-02-23T03:35:49.872 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-23T03:35:49.891 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2022-02-23T03:35:49.891 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-23T03:35:49.908 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2022-02-23T03:35:49.909 INFO:teuthology.lock.ops:New key found. Updating... 2022-02-23T03:35:49.925 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: set os type and version on server 2022-02-23T03:35:49.925 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2022-02-23T03:35:50.106 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2022-02-23T03:35:50.112 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="Ubuntu" 2022-02-23T03:35:50.113 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-23T03:35:50.113 INFO:teuthology.orchestra.run.smithi072.stdout:ID=ubuntu 2022-02-23T03:35:50.113 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE=debian 2022-02-23T03:35:50.114 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-23T03:35:50.114 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="20.04" 2022-02-23T03:35:50.114 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-23T03:35:50.114 INFO:teuthology.orchestra.run.smithi072.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-23T03:35:50.115 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-23T03:35:50.115 INFO:teuthology.orchestra.run.smithi072.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-23T03:35:50.115 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_CODENAME=focal 2022-02-23T03:35:50.115 INFO:teuthology.orchestra.run.smithi072.stdout:UBUNTU_CODENAME=focal 2022-02-23T03:35:50.116 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2022-02-23T03:35:50.135 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2022-02-23T03:35:50.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2022-02-23T03:35:50.394 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2022-02-23T03:35:50.401 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="Ubuntu" 2022-02-23T03:35:50.401 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="20.04.3 LTS (Focal Fossa)" 2022-02-23T03:35:50.401 INFO:teuthology.orchestra.run.smithi159.stdout:ID=ubuntu 2022-02-23T03:35:50.401 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE=debian 2022-02-23T03:35:50.402 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="Ubuntu 20.04.3 LTS" 2022-02-23T03:35:50.402 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="20.04" 2022-02-23T03:35:50.402 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://www.ubuntu.com/" 2022-02-23T03:35:50.402 INFO:teuthology.orchestra.run.smithi159.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2022-02-23T03:35:50.403 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2022-02-23T03:35:50.403 INFO:teuthology.orchestra.run.smithi159.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2022-02-23T03:35:50.403 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_CODENAME=focal 2022-02-23T03:35:50.403 INFO:teuthology.orchestra.run.smithi159.stdout:UBUNTU_CODENAME=focal 2022-02-23T03:35:50.404 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2022-02-23T03:35:50.422 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2022-02-23T03:35:50.423 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2022-02-23T04:36:50.745 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/dispatcher/supervisor.py", line 206, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/lock/ops.py", line 317, in reimage_machines update_nodes(reimaged) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/lock/ops.py", line 38, in update_nodes inventory_info = remote.inventory_info File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/remote.py", line 663, in inventory_info node['arch'] = self.arch File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/remote.py", line 320, in arch self._arch = self.sh('uname -m').strip() File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1010, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel. 2022-02-23T04:36:50.836 INFO:teuthology.nuke:Checking targets against current locks 2022-02-23T04:36:50.966 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-23T04:36:50.970 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-23T04:36:50.972 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-02-23T04:36:51.052 INFO:teuthology.orchestra.console:Power off smithi172 2022-02-23T04:36:51.069 INFO:teuthology.orchestra.console:Power off smithi072 2022-02-23T04:36:51.081 INFO:teuthology.orchestra.console:Power off smithi159 2022-02-23T04:36:59.441 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2022-02-23T04:36:59.449 INFO:teuthology.orchestra.console:Power off for smithi072 completed 2022-02-23T04:36:59.455 INFO:teuthology.orchestra.console:Power off for smithi159 completed 2022-02-23T04:36:59.601 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com 2022-02-23T04:36:59.604 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com 2022-02-23T04:36:59.606 INFO:teuthology.lock.ops:unlocked: smithi159.front.sepia.ceph.com 2022-02-23T04:36:59.905 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/dispatcher/supervisor.py", line 52, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/dispatcher/supervisor.py", line 206, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/lock/ops.py", line 317, in reimage_machines update_nodes(reimaged) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/lock/ops.py", line 38, in update_nodes inventory_info = remote.inventory_info File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/remote.py", line 663, in inventory_info node['arch'] = self.arch File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/remote.py", line 320, in arch self._arch = self.sh('uname -m').strip() File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/remote.py", line 509, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/teuthology/orchestra/run.py", line 100, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_eea2521245e542c7c1a063d296779f572aa0255a/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1010, in open_channel raise SSHException("Timeout opening channel.") paramiko.ssh_exception.SSHException: Timeout opening channel.