2021-12-02T19:50:40.917 INFO:root:teuthology version: 1.1.0-844d1377 2021-12-02T19:50:41.127 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi006.front.sepia.ceph.com' reimaging 2021-12-02T19:50:41.128 INFO:teuthology.lock.ops:Updating [ubuntu@smithi006.front.sepia.ceph.com]: reset os type and version on server 2021-12-02T19:50:41.128 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2021-12-02T19:50:41.149 INFO:teuthology.lock.ops:Node 'ubuntu@smithi006.front.sepia.ceph.com' reimaging is complete 2021-12-02T19:50:41.150 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi025.front.sepia.ceph.com' reimaging 2021-12-02T19:50:41.150 INFO:teuthology.lock.ops:Updating [ubuntu@smithi025.front.sepia.ceph.com]: reset os type and version on server 2021-12-02T19:50:41.151 INFO:teuthology.lock.ops:Updating smithi025.front.sepia.ceph.com on lock server 2021-12-02T19:50:41.169 INFO:teuthology.lock.ops:Node 'ubuntu@smithi025.front.sepia.ceph.com' reimaging is complete 2021-12-02T19:50:41.170 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi073.front.sepia.ceph.com' reimaging 2021-12-02T19:50:41.170 INFO:teuthology.lock.ops:Updating [ubuntu@smithi073.front.sepia.ceph.com]: reset os type and version on server 2021-12-02T19:50:41.171 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2021-12-02T19:50:41.191 INFO:teuthology.lock.ops:Node 'ubuntu@smithi073.front.sepia.ceph.com' reimaging is complete 2021-12-02T19:50:41.191 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi085.front.sepia.ceph.com' reimaging 2021-12-02T19:50:41.192 INFO:teuthology.lock.ops:Updating [ubuntu@smithi085.front.sepia.ceph.com]: reset os type and version on server 2021-12-02T19:50:41.192 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server 2021-12-02T19:50:41.211 INFO:teuthology.lock.ops:Node 'ubuntu@smithi085.front.sepia.ceph.com' reimaging is complete 2021-12-02T19:50:41.380 INFO:teuthology.provision.fog.smithi025:Scheduling deploy of ubuntu 20.04 2021-12-02T19:50:41.394 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of ubuntu 20.04 2021-12-02T19:50:41.417 INFO:teuthology.provision.fog.smithi085:Scheduling deploy of ubuntu 20.04 2021-12-02T19:50:41.427 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of ubuntu 20.04 2021-12-02T19:50:41.743 INFO:teuthology.orchestra.console:Power off smithi006 2021-12-02T19:50:41.758 INFO:teuthology.orchestra.console:Power off smithi073 2021-12-02T19:50:41.818 INFO:teuthology.orchestra.console:Power off smithi025 2021-12-02T19:50:41.857 INFO:teuthology.orchestra.console:Power off smithi085 2021-12-02T19:50:41.896 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2021-12-02T19:50:41.967 INFO:teuthology.orchestra.console:Power off for smithi025 completed 2021-12-02T19:50:42.001 INFO:teuthology.orchestra.console:Power on smithi006 2021-12-02T19:50:42.022 INFO:teuthology.orchestra.console:Power off for smithi085 completed 2021-12-02T19:50:42.069 INFO:teuthology.orchestra.console:Power on smithi025 2021-12-02T19:50:42.124 INFO:teuthology.orchestra.console:Power on smithi085 2021-12-02T19:50:46.302 INFO:teuthology.orchestra.console:Power on for smithi006 completed 2021-12-02T19:50:46.404 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish 2021-12-02T19:50:46.475 INFO:teuthology.orchestra.console:Power on for smithi085 completed 2021-12-02T19:50:46.577 INFO:teuthology.provision.fog.smithi085:Waiting for deploy to finish 2021-12-02T19:50:50.131 INFO:teuthology.orchestra.console:Power off for smithi073 completed 2021-12-02T19:50:50.233 INFO:teuthology.orchestra.console:Power on smithi073 2021-12-02T19:50:50.560 INFO:teuthology.orchestra.console:Power on for smithi025 completed 2021-12-02T19:50:50.661 INFO:teuthology.provision.fog.smithi025:Waiting for deploy to finish 2021-12-02T19:50:54.487 INFO:teuthology.orchestra.console:Power on for smithi073 completed 2021-12-02T19:50:54.589 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish 2021-12-02T19:55:28.352 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-02T19:55:28.478 INFO:teuthology.provision.fog.smithi006:Node is ready 2021-12-02T19:55:28.485 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006 2021-12-02T19:55:28.534 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006 smithi006 2021-12-02T19:55:28.651 INFO:teuthology.provision.fog.smithi006:Deploy complete! 2021-12-02T19:55:54.777 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:55:55.836 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:55:56.896 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:55:57.954 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:55:59.058 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:00.157 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:01.275 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:02.375 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:03.474 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:04.574 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:10.578 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-12-02T19:56:10.579 ERROR:paramiko.transport:Traceback (most recent call last): 2021-12-02T19:56:10.579 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-12-02T19:56:10.580 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-12-02T19:56:10.580 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-12-02T19:56:10.580 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-12-02T19:56:10.581 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-12-02T19:56:10.581 ERROR:paramiko.transport: raise EOFError() 2021-12-02T19:56:10.581 ERROR:paramiko.transport:EOFError 2021-12-02T19:56:10.582 ERROR:paramiko.transport: 2021-12-02T19:56:10.582 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-12-02T19:56:10.583 ERROR:paramiko.transport: 2021-12-02T19:56:10.583 ERROR:paramiko.transport:Traceback (most recent call last): 2021-12-02T19:56:10.583 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-12-02T19:56:10.584 ERROR:paramiko.transport: self._check_banner() 2021-12-02T19:56:10.584 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-12-02T19:56:10.584 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-12-02T19:56:10.584 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-12-02T19:56:10.585 ERROR:paramiko.transport: 2021-12-02T19:56:16.682 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:17.684 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-12-02T19:56:17.684 ERROR:paramiko.transport:Traceback (most recent call last): 2021-12-02T19:56:17.685 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-12-02T19:56:17.685 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-12-02T19:56:17.685 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-12-02T19:56:17.686 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-12-02T19:56:17.686 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-12-02T19:56:17.686 ERROR:paramiko.transport: raise EOFError() 2021-12-02T19:56:17.687 ERROR:paramiko.transport:EOFError 2021-12-02T19:56:17.687 ERROR:paramiko.transport: 2021-12-02T19:56:17.687 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-12-02T19:56:17.688 ERROR:paramiko.transport: 2021-12-02T19:56:17.688 ERROR:paramiko.transport:Traceback (most recent call last): 2021-12-02T19:56:17.688 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-12-02T19:56:17.689 ERROR:paramiko.transport: self._check_banner() 2021-12-02T19:56:17.689 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-12-02T19:56:17.689 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-12-02T19:56:17.689 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-12-02T19:56:17.690 ERROR:paramiko.transport: 2021-12-02T19:56:21.424 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-02T19:56:21.970 INFO:teuthology.provision.fog.smithi085:Node is ready 2021-12-02T19:56:21.976 INFO:teuthology.orchestra.run.smithi085.stdout:smithi085.front.sepia.ceph.com 2021-12-02T19:56:21.984 INFO:teuthology.orchestra.run.smithi085.stdout:172.21.15.85 smithi085.front.sepia.ceph.com smithi085 2021-12-02T19:56:22.195 INFO:teuthology.provision.fog.smithi085:Deploy complete! 2021-12-02T19:56:23.748 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:24.844 ERROR:teuthology.orchestra.connection:Error authenticating with smithi025.front.sepia.ceph.com: Authentication failed. 2021-12-02T19:56:29.250 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-02T19:56:29.808 INFO:teuthology.provision.fog.smithi073:Node is ready 2021-12-02T19:56:29.815 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073.front.sepia.ceph.com 2021-12-02T19:56:29.865 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73 smithi073.front.sepia.ceph.com smithi073 2021-12-02T19:56:29.968 INFO:teuthology.provision.fog.smithi073:Deploy complete! 2021-12-02T19:59:15.891 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-12-02T19:59:16.187 INFO:teuthology.provision.fog.smithi025:Node is ready 2021-12-02T19:59:16.203 INFO:teuthology.orchestra.run.smithi025.stdout:smithi025.front.sepia.ceph.com 2021-12-02T19:59:16.258 INFO:teuthology.orchestra.run.smithi025.stdout:172.21.15.25 smithi025.front.sepia.ceph.com smithi025 2021-12-02T19:59:16.596 INFO:teuthology.provision.fog.smithi025:Deploy complete! 2021-12-02T19:59:25.739 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/lock/ops.py", line 316, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/lock/ops.py", line 283, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi006.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds 2021-12-02T19:59:25.803 INFO:teuthology.nuke:Checking targets against current locks 2021-12-02T19:59:25.958 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-02T19:59:25.963 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-02T19:59:25.964 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-02T19:59:25.966 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-12-02T19:59:26.053 INFO:teuthology.orchestra.console:Power off smithi085 2021-12-02T19:59:26.069 INFO:teuthology.orchestra.console:Power off smithi006 2021-12-02T19:59:26.080 INFO:teuthology.orchestra.console:Power off smithi025 2021-12-02T19:59:26.094 INFO:teuthology.orchestra.console:Power off smithi073 2021-12-02T19:59:34.456 INFO:teuthology.orchestra.console:Power off for smithi006 completed 2021-12-02T19:59:34.604 INFO:teuthology.lock.ops:unlocked: smithi006.front.sepia.ceph.com 2021-12-02T19:59:38.574 INFO:teuthology.orchestra.console:Power off for smithi025 completed 2021-12-02T19:59:38.582 INFO:teuthology.orchestra.console:Power off for smithi085 completed 2021-12-02T19:59:38.590 INFO:teuthology.orchestra.console:Power off for smithi073 completed 2021-12-02T19:59:38.718 INFO:teuthology.lock.ops:unlocked: smithi025.front.sepia.ceph.com 2021-12-02T19:59:38.729 INFO:teuthology.lock.ops:unlocked: smithi085.front.sepia.ceph.com 2021-12-02T19:59:38.732 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com 2021-12-02T19:59:38.733 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/dispatcher/supervisor.py", line 50, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/lock/ops.py", line 316, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/lock/ops.py", line 283, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_844d13778e221f3c1f3cb6626445c4bc2f71766d/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi006.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds