2021-06-23T10:48:19.104 INFO:root:teuthology version: 1.1.0-26631ede 2021-06-23T10:48:19.280 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi028.front.sepia.ceph.com' reimaging 2021-06-23T10:48:19.282 INFO:teuthology.lock.ops:Updating [ubuntu@smithi028.front.sepia.ceph.com]: reset os type and version on server 2021-06-23T10:48:19.282 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2021-06-23T10:48:19.300 INFO:teuthology.lock.ops:Node 'ubuntu@smithi028.front.sepia.ceph.com' reimaging is complete 2021-06-23T10:48:19.301 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi033.front.sepia.ceph.com' reimaging 2021-06-23T10:48:19.301 INFO:teuthology.lock.ops:Updating [ubuntu@smithi033.front.sepia.ceph.com]: reset os type and version on server 2021-06-23T10:48:19.302 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2021-06-23T10:48:19.320 INFO:teuthology.lock.ops:Node 'ubuntu@smithi033.front.sepia.ceph.com' reimaging is complete 2021-06-23T10:48:19.320 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi051.front.sepia.ceph.com' reimaging 2021-06-23T10:48:19.321 INFO:teuthology.lock.ops:Updating [ubuntu@smithi051.front.sepia.ceph.com]: reset os type and version on server 2021-06-23T10:48:19.321 INFO:teuthology.lock.ops:Updating smithi051.front.sepia.ceph.com on lock server 2021-06-23T10:48:19.340 INFO:teuthology.lock.ops:Node 'ubuntu@smithi051.front.sepia.ceph.com' reimaging is complete 2021-06-23T10:48:19.585 INFO:teuthology.provision.fog.smithi028:Scheduling deploy of ubuntu 18.04 2021-06-23T10:48:19.599 INFO:teuthology.provision.fog.smithi051:Scheduling deploy of ubuntu 18.04 2021-06-23T10:48:19.664 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of ubuntu 18.04 2021-06-23T10:48:20.362 INFO:teuthology.orchestra.console:Power off smithi051 2021-06-23T10:48:20.403 INFO:teuthology.orchestra.console:Power off smithi028 2021-06-23T10:48:20.450 INFO:teuthology.orchestra.console:Power off smithi033 2021-06-23T10:48:20.570 INFO:teuthology.orchestra.console:Power off for smithi051 completed 2021-06-23T10:48:20.672 INFO:teuthology.orchestra.console:Power on smithi051 2021-06-23T10:48:20.681 INFO:teuthology.orchestra.console:Power off for smithi028 completed 2021-06-23T10:48:20.706 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2021-06-23T10:48:20.782 INFO:teuthology.orchestra.console:Power on smithi028 2021-06-23T10:48:20.814 INFO:teuthology.orchestra.console:Power on smithi033 2021-06-23T10:48:24.947 INFO:teuthology.orchestra.console:Power on for smithi051 completed 2021-06-23T10:48:25.049 INFO:teuthology.provision.fog.smithi051:Waiting for deploy to finish 2021-06-23T10:48:25.072 INFO:teuthology.orchestra.console:Power on for smithi033 completed 2021-06-23T10:48:25.173 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2021-06-23T10:48:29.212 INFO:teuthology.orchestra.console:Power on for smithi028 completed 2021-06-23T10:48:29.314 INFO:teuthology.provision.fog.smithi028:Waiting for deploy to finish 2021-06-23T10:52:22.859 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:52:28.876 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:52:31.044 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:52:34.890 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:52:37.058 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:52:40.900 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:52:43.071 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:52:46.988 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-23T10:52:47.290 INFO:teuthology.provision.fog.smithi028:Node is ready 2021-06-23T10:52:47.337 INFO:teuthology.orchestra.run.smithi028.stdout:smithi028.front.sepia.ceph.com 2021-06-23T10:52:47.387 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28 smithi028.front.sepia.ceph.com smithi028 2021-06-23T10:52:47.788 INFO:teuthology.provision.fog.smithi028:Deploy complete! 2021-06-23T10:52:49.085 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:52:55.171 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-23T10:52:55.520 INFO:teuthology.provision.fog.smithi033:Node is ready 2021-06-23T10:52:55.568 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com 2021-06-23T10:52:55.703 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033 2021-06-23T10:52:56.128 INFO:teuthology.provision.fog.smithi033:Deploy complete! 2021-06-23T10:53:38.520 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:53:44.531 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:53:50.544 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T10:53:56.633 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-23T10:53:57.017 INFO:teuthology.provision.fog.smithi051:Node is ready 2021-06-23T10:53:57.064 INFO:teuthology.orchestra.run.smithi051.stdout:smithi051.front.sepia.ceph.com 2021-06-23T10:53:57.113 INFO:teuthology.orchestra.run.smithi051.stdout:172.21.15.51 smithi051.front.sepia.ceph.com smithi051 2021-06-23T10:53:57.431 INFO:teuthology.provision.fog.smithi051:Deploy complete! 2021-06-23T10:54:06.547 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi028.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds 2021-06-23T10:54:07.650 INFO:teuthology.nuke:Checking targets against current locks 2021-06-23T10:54:07.738 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-23T10:54:07.742 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-23T10:54:07.743 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-23T10:54:07.805 INFO:teuthology.orchestra.console:Power off smithi033 2021-06-23T10:54:07.821 INFO:teuthology.orchestra.console:Power off smithi028 2021-06-23T10:54:07.834 INFO:teuthology.orchestra.console:Power off smithi051 2021-06-23T10:54:12.105 INFO:teuthology.orchestra.console:Power off for smithi028 completed 2021-06-23T10:54:12.248 INFO:teuthology.lock.ops:unlocked smithi028.front.sepia.ceph.com 2021-06-23T10:54:16.210 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2021-06-23T10:54:16.356 INFO:teuthology.lock.ops:unlocked smithi033.front.sepia.ceph.com 2021-06-23T10:54:20.338 INFO:teuthology.orchestra.console:Power off for smithi051 completed 2021-06-23T10:54:20.479 INFO:teuthology.lock.ops:unlocked smithi051.front.sepia.ceph.com 2021-06-23T10:54:20.480 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/dispatcher/supervisor.py", line 50, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_26631ede9027a5643f2e47dacc5af6f1834aac45/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi028.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds