2021-04-09T18:39:44.842 INFO:root:teuthology version: 1.1.0-85d61eae 2021-04-09T18:39:45.001 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi058.front.sepia.ceph.com' reimaging 2021-04-09T18:39:45.002 INFO:teuthology.lock.ops:Updating [ubuntu@smithi058.front.sepia.ceph.com]: reset os type and version on server 2021-04-09T18:39:45.003 INFO:teuthology.lock.ops:Updating smithi058.front.sepia.ceph.com on lock server 2021-04-09T18:39:45.018 INFO:teuthology.lock.ops:Node 'ubuntu@smithi058.front.sepia.ceph.com' reimaging is complete 2021-04-09T18:39:45.019 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging 2021-04-09T18:39:45.020 INFO:teuthology.lock.ops:Updating [ubuntu@smithi120.front.sepia.ceph.com]: reset os type and version on server 2021-04-09T18:39:45.020 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2021-04-09T18:39:45.037 INFO:teuthology.lock.ops:Node 'ubuntu@smithi120.front.sepia.ceph.com' reimaging is complete 2021-04-09T18:39:45.037 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi132.front.sepia.ceph.com' reimaging 2021-04-09T18:39:45.037 INFO:teuthology.lock.ops:Updating [ubuntu@smithi132.front.sepia.ceph.com]: reset os type and version on server 2021-04-09T18:39:45.038 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2021-04-09T18:39:45.052 INFO:teuthology.lock.ops:Node 'ubuntu@smithi132.front.sepia.ceph.com' reimaging is complete 2021-04-09T18:39:45.053 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi161.front.sepia.ceph.com' reimaging 2021-04-09T18:39:45.053 INFO:teuthology.lock.ops:Updating [ubuntu@smithi161.front.sepia.ceph.com]: reset os type and version on server 2021-04-09T18:39:45.053 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2021-04-09T18:39:45.068 INFO:teuthology.lock.ops:Node 'ubuntu@smithi161.front.sepia.ceph.com' reimaging is complete 2021-04-09T18:39:48.684 INFO:teuthology.provision.fog.smithi058:Scheduling deploy of ubuntu 18.04 2021-04-09T18:39:48.904 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of ubuntu 18.04 2021-04-09T18:39:48.953 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of ubuntu 18.04 2021-04-09T18:39:49.198 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of ubuntu 18.04 2021-04-09T18:39:57.866 INFO:teuthology.orchestra.console:Power off smithi058 2021-04-09T18:39:57.995 INFO:teuthology.orchestra.console:Power off smithi161 2021-04-09T18:39:58.006 INFO:teuthology.orchestra.console:Power off for smithi058 completed 2021-04-09T18:39:58.108 INFO:teuthology.orchestra.console:Power on smithi058 2021-04-09T18:39:58.135 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2021-04-09T18:39:58.236 INFO:teuthology.orchestra.console:Power on smithi161 2021-04-09T18:40:06.467 INFO:teuthology.orchestra.console:Power on for smithi058 completed 2021-04-09T18:40:06.569 INFO:teuthology.provision.fog.smithi058:Waiting for deploy to finish 2021-04-09T18:40:06.591 INFO:teuthology.orchestra.console:Power on for smithi161 completed 2021-04-09T18:40:06.692 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish 2021-04-09T18:40:17.320 INFO:teuthology.orchestra.console:Power off smithi120 2021-04-09T18:40:17.351 INFO:teuthology.orchestra.console:Power off smithi132 2021-04-09T18:40:17.460 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2021-04-09T18:40:17.491 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2021-04-09T18:40:17.562 INFO:teuthology.orchestra.console:Power on smithi120 2021-04-09T18:40:17.592 INFO:teuthology.orchestra.console:Power on smithi132 2021-04-09T18:40:25.923 INFO:teuthology.orchestra.console:Power on for smithi120 completed 2021-04-09T18:40:25.946 INFO:teuthology.orchestra.console:Power on for smithi132 completed 2021-04-09T18:40:26.025 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish 2021-04-09T18:40:26.048 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2021-04-09T18:43:31.613 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-04-09T18:43:31.614 ERROR:paramiko.transport:Traceback (most recent call last): 2021-04-09T18:43:31.615 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-04-09T18:43:31.615 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-04-09T18:43:31.616 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-04-09T18:43:31.616 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-04-09T18:43:31.616 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-04-09T18:43:31.617 ERROR:paramiko.transport: raise EOFError() 2021-04-09T18:43:31.618 ERROR:paramiko.transport:EOFError 2021-04-09T18:43:31.618 ERROR:paramiko.transport: 2021-04-09T18:43:31.619 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-04-09T18:43:31.619 ERROR:paramiko.transport: 2021-04-09T18:43:31.620 ERROR:paramiko.transport:Traceback (most recent call last): 2021-04-09T18:43:31.620 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-04-09T18:43:31.620 ERROR:paramiko.transport: self._check_banner() 2021-04-09T18:43:31.620 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-04-09T18:43:31.621 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-04-09T18:43:31.621 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-04-09T18:43:31.622 ERROR:paramiko.transport: 2021-04-09T18:44:25.161 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-04-09T18:44:31.174 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-04-09T18:44:37.186 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-04-09T18:44:43.200 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-04-09T18:44:49.277 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-09T18:44:49.468 INFO:teuthology.provision.fog.smithi058:Node is ready 2021-04-09T18:44:49.517 INFO:teuthology.orchestra.run.smithi058.stdout:smithi058 2021-04-09T18:44:49.565 INFO:teuthology.orchestra.run.smithi058.stdout:172.21.15.58 smithi058 smithi058 2021-04-09T18:44:49.846 INFO:teuthology.provision.fog.smithi058:Deploy complete! 2021-04-09T18:45:28.167 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-04-09T18:45:34.309 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-09T18:45:34.676 INFO:teuthology.provision.fog.smithi120:Node is ready 2021-04-09T18:45:34.723 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com 2021-04-09T18:45:34.772 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120 2021-04-09T18:45:35.141 INFO:teuthology.provision.fog.smithi120:Deploy complete! 2021-04-09T18:48:17.424 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-04-09T18:48:23.500 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-09T18:48:23.880 INFO:teuthology.provision.fog.smithi132:Node is ready 2021-04-09T18:48:23.933 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com 2021-04-09T18:48:23.985 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132 2021-04-09T18:48:24.322 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2021-04-09T18:48:59.855 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-04-09T18:49:00.045 INFO:teuthology.provision.fog.smithi161:Node is ready 2021-04-09T18:49:00.092 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161 2021-04-09T18:49:00.140 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161 smithi161 2021-04-09T18:49:00.423 INFO:teuthology.provision.fog.smithi161:Deploy complete! 2021-04-09T18:49:09.545 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi058.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds 2021-04-09T18:49:09.600 INFO:teuthology.nuke:Checking targets against current locks 2021-04-09T18:49:09.714 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-09T18:49:09.727 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-09T18:49:09.738 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-09T18:49:09.750 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-04-09T18:49:10.000 INFO:teuthology.orchestra.console:Power off smithi132 2021-04-09T18:49:10.024 INFO:teuthology.orchestra.console:Power off smithi120 2021-04-09T18:49:10.041 INFO:teuthology.orchestra.console:Power off smithi058 2021-04-09T18:49:10.071 INFO:teuthology.orchestra.console:Power off smithi161 2021-04-09T18:49:18.426 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2021-04-09T18:49:18.453 INFO:teuthology.orchestra.console:Power off for smithi058 completed 2021-04-09T18:49:18.635 INFO:teuthology.lock.ops:unlocked smithi132.front.sepia.ceph.com 2021-04-09T18:49:18.655 INFO:teuthology.lock.ops:unlocked smithi058.front.sepia.ceph.com 2021-04-09T18:49:22.557 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2021-04-09T18:49:22.587 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2021-04-09T18:49:22.789 INFO:teuthology.lock.ops:unlocked smithi120.front.sepia.ceph.com 2021-04-09T18:49:22.815 INFO:teuthology.lock.ops:unlocked smithi161.front.sepia.ceph.com 2021-04-09T18:49:22.816 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 50, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_85d61eae4759f46ce21e9a37cd816a7a1a66c9d5/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi058.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds