2021-06-23T05:28:39.028 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-23T05:28:39.116 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba014.front.sepia.ceph.com' reimaging 2021-06-23T05:28:39.117 INFO:teuthology.lock.ops:Updating [ubuntu@gibba014.front.sepia.ceph.com]: reset os type and version on server 2021-06-23T05:28:39.117 INFO:teuthology.lock.ops:Updating gibba014.front.sepia.ceph.com on lock server 2021-06-23T05:28:39.129 INFO:teuthology.lock.ops:Node 'ubuntu@gibba014.front.sepia.ceph.com' reimaging is complete 2021-06-23T05:28:39.130 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba037.front.sepia.ceph.com' reimaging 2021-06-23T05:28:39.130 INFO:teuthology.lock.ops:Updating [ubuntu@gibba037.front.sepia.ceph.com]: reset os type and version on server 2021-06-23T05:28:39.131 INFO:teuthology.lock.ops:Updating gibba037.front.sepia.ceph.com on lock server 2021-06-23T05:28:39.144 INFO:teuthology.lock.ops:Node 'ubuntu@gibba037.front.sepia.ceph.com' reimaging is complete 2021-06-23T05:28:39.329 INFO:teuthology.provision.fog.gibba014:Scheduling deploy of ubuntu 18.04 2021-06-23T05:28:39.351 INFO:teuthology.provision.fog.gibba037:Scheduling deploy of ubuntu 18.04 2021-06-23T05:28:39.725 INFO:teuthology.orchestra.console:Power off gibba014 2021-06-23T05:28:39.765 INFO:teuthology.orchestra.console:Power off gibba037 2021-06-23T05:28:39.874 INFO:teuthology.orchestra.console:Power off for gibba014 completed 2021-06-23T05:28:39.910 INFO:teuthology.orchestra.console:Power off for gibba037 completed 2021-06-23T05:28:39.976 INFO:teuthology.orchestra.console:Power on gibba014 2021-06-23T05:28:40.012 INFO:teuthology.orchestra.console:Power on gibba037 2021-06-23T05:28:48.346 INFO:teuthology.orchestra.console:Power on for gibba014 completed 2021-06-23T05:28:48.377 INFO:teuthology.orchestra.console:Power on for gibba037 completed 2021-06-23T05:28:48.448 INFO:teuthology.provision.fog.gibba014:Waiting for deploy to finish 2021-06-23T05:28:48.478 INFO:teuthology.provision.fog.gibba037:Waiting for deploy to finish 2021-06-23T05:32:59.902 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:00.998 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:02.092 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:03.187 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:04.281 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:05.376 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:06.470 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:07.524 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:08.578 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:09.672 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:12.133 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T05:33:15.675 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-06-23T05:33:15.676 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T05:33:15.676 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-06-23T05:33:15.676 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-06-23T05:33:15.677 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-06-23T05:33:15.677 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-06-23T05:33:15.677 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-06-23T05:33:15.677 ERROR:paramiko.transport: raise EOFError() 2021-06-23T05:33:15.678 ERROR:paramiko.transport:EOFError 2021-06-23T05:33:15.678 ERROR:paramiko.transport: 2021-06-23T05:33:15.678 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-06-23T05:33:15.679 ERROR:paramiko.transport: 2021-06-23T05:33:15.679 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T05:33:15.679 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-06-23T05:33:15.680 ERROR:paramiko.transport: self._check_banner() 2021-06-23T05:33:15.680 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-06-23T05:33:15.680 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-06-23T05:33:15.681 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-06-23T05:33:15.681 ERROR:paramiko.transport: 2021-06-23T05:33:18.144 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2021-06-23T05:33:21.684 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-06-23T05:33:21.684 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T05:33:21.685 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-06-23T05:33:21.685 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-06-23T05:33:21.685 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-06-23T05:33:21.685 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-06-23T05:33:21.686 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-06-23T05:33:21.686 ERROR:paramiko.transport: raise EOFError() 2021-06-23T05:33:21.686 ERROR:paramiko.transport:EOFError 2021-06-23T05:33:21.687 ERROR:paramiko.transport: 2021-06-23T05:33:21.687 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-06-23T05:33:21.687 ERROR:paramiko.transport: 2021-06-23T05:33:21.687 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T05:33:21.688 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-06-23T05:33:21.688 ERROR:paramiko.transport: self._check_banner() 2021-06-23T05:33:21.688 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-06-23T05:33:21.689 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-06-23T05:33:21.689 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-06-23T05:33:21.689 ERROR:paramiko.transport: 2021-06-23T05:33:27.743 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:28.837 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:29.932 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:31.027 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:32.121 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:33.215 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:33.262 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-23T05:33:33.771 INFO:teuthology.provision.fog.gibba014:Node is ready 2021-06-23T05:33:33.820 INFO:teuthology.orchestra.run.gibba014.stdout:gibba014.front.sepia.ceph.com 2021-06-23T05:33:33.869 INFO:teuthology.orchestra.run.gibba014.stdout:172.21.2.114 gibba014.front.sepia.ceph.com gibba014 2021-06-23T05:33:34.190 INFO:teuthology.provision.fog.gibba014:Deploy complete! 2021-06-23T05:33:34.310 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:35.313 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-06-23T05:33:35.313 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T05:33:35.314 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-06-23T05:33:35.314 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-06-23T05:33:35.314 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-06-23T05:33:35.315 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-06-23T05:33:35.315 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-06-23T05:33:35.315 ERROR:paramiko.transport: raise EOFError() 2021-06-23T05:33:35.316 ERROR:paramiko.transport:EOFError 2021-06-23T05:33:35.316 ERROR:paramiko.transport: 2021-06-23T05:33:35.316 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-06-23T05:33:35.317 ERROR:paramiko.transport: 2021-06-23T05:33:35.317 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T05:33:35.317 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-06-23T05:33:35.317 ERROR:paramiko.transport: self._check_banner() 2021-06-23T05:33:35.318 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-06-23T05:33:35.318 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-06-23T05:33:35.318 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-06-23T05:33:35.319 ERROR:paramiko.transport: 2021-06-23T05:33:41.413 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:42.507 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:33:43.602 ERROR:teuthology.orchestra.connection:Error authenticating with gibba037.front.sepia.ceph.com: Authentication failed. 2021-06-23T05:36:05.744 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-23T05:36:05.823 INFO:teuthology.provision.fog.gibba037:Node is ready 2021-06-23T05:36:05.875 INFO:teuthology.orchestra.run.gibba037.stdout:gibba037 2021-06-23T05:36:05.927 INFO:teuthology.orchestra.run.gibba037.stdout:172.21.2.137 gibba037 gibba037 2021-06-23T05:36:06.226 INFO:teuthology.provision.fog.gibba037:Deploy complete! 2021-06-23T05:36:12.747 ERROR:teuthology.misc:write (gibba014.front.sepia.ceph.com): No route to host 2021-06-23T05:36:14.756 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan gibba014.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds 2021-06-23T05:36:14.802 INFO:teuthology.nuke:Checking targets against current locks 2021-06-23T05:36:14.886 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-23T05:36:14.889 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-23T05:36:14.940 INFO:teuthology.orchestra.console:Power off gibba014 2021-06-23T05:36:14.954 INFO:teuthology.orchestra.console:Power off gibba037 2021-06-23T05:36:19.204 INFO:teuthology.orchestra.console:Power off for gibba014 completed 2021-06-23T05:36:19.274 INFO:teuthology.orchestra.console:Power off for gibba037 completed 2021-06-23T05:36:19.343 INFO:teuthology.lock.ops:unlocked gibba014.front.sepia.ceph.com 2021-06-23T05:36:19.411 INFO:teuthology.lock.ops:unlocked gibba037.front.sepia.ceph.com 2021-06-23T05:36:19.412 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 50, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan gibba014.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds