2021-05-12T22:47:56.631 INFO:root:teuthology version: 1.1.0-19220a3b 2021-05-12T22:47:56.759 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi019.front.sepia.ceph.com' reimaging 2021-05-12T22:47:56.761 INFO:teuthology.lock.ops:Updating [ubuntu@smithi019.front.sepia.ceph.com]: reset os type and version on server 2021-05-12T22:47:56.761 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2021-05-12T22:47:56.780 INFO:teuthology.lock.ops:Node 'ubuntu@smithi019.front.sepia.ceph.com' reimaging is complete 2021-05-12T22:47:56.780 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi050.front.sepia.ceph.com' reimaging 2021-05-12T22:47:56.780 INFO:teuthology.lock.ops:Updating [ubuntu@smithi050.front.sepia.ceph.com]: reset os type and version on server 2021-05-12T22:47:56.781 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2021-05-12T22:47:56.797 INFO:teuthology.lock.ops:Node 'ubuntu@smithi050.front.sepia.ceph.com' reimaging is complete 2021-05-12T22:47:56.798 INFO:teuthology.lock.ops:Start node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging 2021-05-12T22:47:56.798 INFO:teuthology.lock.ops:Updating [ubuntu@smithi179.front.sepia.ceph.com]: reset os type and version on server 2021-05-12T22:47:56.798 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2021-05-12T22:47:56.827 INFO:teuthology.lock.ops:Node 'ubuntu@smithi179.front.sepia.ceph.com' reimaging is complete 2021-05-12T22:47:57.045 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of ubuntu 18.04 2021-05-12T22:47:57.052 INFO:teuthology.provision.fog.smithi179:Scheduling deploy of ubuntu 18.04 2021-05-12T22:47:57.082 INFO:teuthology.provision.fog.smithi019:Scheduling deploy of ubuntu 18.04 2021-05-12T22:47:57.545 INFO:teuthology.orchestra.console:Power off smithi019 2021-05-12T22:47:57.606 INFO:teuthology.orchestra.console:Power off smithi050 2021-05-12T22:47:57.641 INFO:teuthology.orchestra.console:Power off smithi179 2021-05-12T22:47:57.687 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2021-05-12T22:47:57.751 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2021-05-12T22:47:57.783 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2021-05-12T22:47:57.788 INFO:teuthology.orchestra.console:Power on smithi019 2021-05-12T22:47:57.854 INFO:teuthology.orchestra.console:Power on smithi050 2021-05-12T22:47:57.884 INFO:teuthology.orchestra.console:Power on smithi179 2021-05-12T22:48:02.136 INFO:teuthology.orchestra.console:Power on for smithi179 completed 2021-05-12T22:48:02.237 INFO:teuthology.provision.fog.smithi179:Waiting for deploy to finish 2021-05-12T22:48:06.158 INFO:teuthology.orchestra.console:Power on for smithi019 completed 2021-05-12T22:48:06.259 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2021-05-12T22:48:06.259 INFO:teuthology.provision.fog.smithi019:Waiting for deploy to finish 2021-05-12T22:48:06.360 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2021-05-12T22:50:37.821 ERROR:teuthology.orchestra.connection:Error authenticating with smithi019.front.sepia.ceph.com: Authentication failed. 2021-05-12T22:52:02.946 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-05-12T22:52:04.872 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-05-12T22:52:08.959 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-05-12T22:52:10.884 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-05-12T22:52:14.971 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-05-12T22:52:16.897 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-05-12T22:52:20.984 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-05-12T22:52:22.909 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-05-12T22:52:27.065 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-12T22:52:27.464 INFO:teuthology.provision.fog.smithi019:Node is ready 2021-05-12T22:52:27.512 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019.front.sepia.ceph.com 2021-05-12T22:52:27.561 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19 smithi019.front.sepia.ceph.com smithi019 2021-05-12T22:52:28.034 INFO:teuthology.provision.fog.smithi019:Deploy complete! 2021-05-12T22:52:29.045 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-12T22:52:29.468 INFO:teuthology.provision.fog.smithi050:Node is ready 2021-05-12T22:52:29.518 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2021-05-12T22:52:29.566 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2021-05-12T22:52:29.882 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2021-05-12T22:53:04.548 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-05-12T22:53:04.933 INFO:teuthology.provision.fog.smithi179:Node is ready 2021-05-12T22:53:04.983 INFO:teuthology.orchestra.run.smithi179.stdout:smithi179.front.sepia.ceph.com 2021-05-12T22:53:05.031 INFO:teuthology.orchestra.run.smithi179.stdout:172.21.15.179 smithi179.front.sepia.ceph.com smithi179 2021-05-12T22:53:05.371 INFO:teuthology.provision.fog.smithi179:Deploy complete! 2021-05-12T22:53:14.514 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi050.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds 2021-05-12T22:53:14.561 INFO:teuthology.nuke:Checking targets against current locks 2021-05-12T22:53:14.676 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-12T22:53:14.680 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-12T22:53:14.682 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-05-12T22:53:14.703 ERROR:teuthology.nuke:Could not nuke {'ubuntu@smithi050.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwYSyRlqhC+v6woR4UE6hCIB2ZyC47+a7qYU1hPReNIqxvCtkIAz4NdULj5N1HO22Qc+6j/iMNzei7zUv5oJ6vKe9/80EEg/hK1l7XtYB3mI4jN+ihJnaX1P4nGX7TpHNtYJqnOcmpgpDUZBd9zz08j22+31ghvS8r5J6u3JP5mQN4EuaoW/4IrvFMyOqkOJAXrs5oRxy3xi40gxpfOqmCa3VpP6NsXkpP7VRGWLzIyQqydzuZ6hyaoVACW6HRltiaK6gr8dXRTBDVM6pzI/+ymrIe6Py7Sv+P+fbsykplk8T0n3OlyFqGI1DiIXYgAT/F0+8uXPuNghOdCPkp48VPF+qpH/Bi/rOSeVTcPKMA4QognLHrWhtXyCJDmB3GRJFyBiiLa4qmKn5Pb+zUqAcRJTpJ4pWPjn2foKm3Fq+mmacF5vd/Lheel44/tiCAXGtk2V2F6Q91gDO4gLppqESKLcsc4BuIljflH/uVObURwkUlHfi8XKTKdKsxDW+eR8='} Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/nuke/__init__.py", line 292, in nuke_one nuke_helper(ctx, should_unlock, keep_logs, should_reboot) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/nuke/__init__.py", line 315, in nuke_helper check_lock.check_lock(ctx, None, check_up=False) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/task/internal/check_lock.py", line 29, in check_lock 'machine {name} is not locked'.format(name=machine) AssertionError: machine ubuntu@smithi050.front.sepia.ceph.com is not locked 2021-05-12T22:53:14.741 INFO:teuthology.orchestra.console:Power off smithi019 2021-05-12T22:53:14.757 INFO:teuthology.orchestra.console:Power off smithi179 2021-05-12T22:53:27.232 INFO:teuthology.orchestra.console:Power off for smithi019 completed 2021-05-12T22:53:27.249 INFO:teuthology.orchestra.console:Power off for smithi179 completed 2021-05-12T22:53:27.375 INFO:teuthology.lock.ops:unlocked smithi019.front.sepia.ceph.com 2021-05-12T22:53:27.390 INFO:teuthology.lock.ops:unlocked smithi179.front.sepia.ceph.com 2021-05-12T22:53:27.392 ERROR:teuthology.nuke:Could not nuke the following targets: targets: ubuntu@smithi050.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYwYSyRlqhC+v6woR4UE6hCIB2ZyC47+a7qYU1hPReNIqxvCtkIAz4NdULj5N1HO22Qc+6j/iMNzei7zUv5oJ6vKe9/80EEg/hK1l7XtYB3mI4jN+ihJnaX1P4nGX7TpHNtYJqnOcmpgpDUZBd9zz08j22+31ghvS8r5J6u3JP5mQN4EuaoW/4IrvFMyOqkOJAXrs5oRxy3xi40gxpfOqmCa3VpP6NsXkpP7VRGWLzIyQqydzuZ6hyaoVACW6HRltiaK6gr8dXRTBDVM6pzI/+ymrIe6Py7Sv+P+fbsykplk8T0n3OlyFqGI1DiIXYgAT/F0+8uXPuNghOdCPkp48VPF+qpH/Bi/rOSeVTcPKMA4QognLHrWhtXyCJDmB3GRJFyBiiLa4qmKn5Pb+zUqAcRJTpJ4pWPjn2foKm3Fq+mmacF5vd/Lheel44/tiCAXGtk2V2F6Q91gDO4gLppqESKLcsc4BuIljflH/uVObURwkUlHfi8XKTKdKsxDW+eR8= 2021-05-12T22:53:27.393 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 50, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_19220a3bd6e252c6e8260827019668a766d85490/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi050.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds