2021-06-23T12:34:48.839 INFO:root:teuthology version: 1.1.0-f359b10d 2021-06-23T12:34:48.938 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba010.front.sepia.ceph.com' reimaging 2021-06-23T12:34:48.939 INFO:teuthology.lock.ops:Updating [ubuntu@gibba010.front.sepia.ceph.com]: reset os type and version on server 2021-06-23T12:34:48.939 INFO:teuthology.lock.ops:Updating gibba010.front.sepia.ceph.com on lock server 2021-06-23T12:34:48.954 INFO:teuthology.lock.ops:Node 'ubuntu@gibba010.front.sepia.ceph.com' reimaging is complete 2021-06-23T12:34:48.954 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba035.front.sepia.ceph.com' reimaging 2021-06-23T12:34:48.955 INFO:teuthology.lock.ops:Updating [ubuntu@gibba035.front.sepia.ceph.com]: reset os type and version on server 2021-06-23T12:34:48.955 INFO:teuthology.lock.ops:Updating gibba035.front.sepia.ceph.com on lock server 2021-06-23T12:34:48.970 INFO:teuthology.lock.ops:Node 'ubuntu@gibba035.front.sepia.ceph.com' reimaging is complete 2021-06-23T12:34:49.160 INFO:teuthology.provision.fog.gibba010:Scheduling deploy of centos 8.3 2021-06-23T12:34:49.176 INFO:teuthology.provision.fog.gibba035:Scheduling deploy of centos 8.3 2021-06-23T12:34:49.620 INFO:teuthology.orchestra.console:Power off gibba035 2021-06-23T12:34:49.690 INFO:teuthology.orchestra.console:Power off gibba010 2021-06-23T12:34:49.774 INFO:teuthology.orchestra.console:Power off for gibba035 completed 2021-06-23T12:34:49.838 INFO:teuthology.orchestra.console:Power off for gibba010 completed 2021-06-23T12:34:49.876 INFO:teuthology.orchestra.console:Power on gibba035 2021-06-23T12:34:49.939 INFO:teuthology.orchestra.console:Power on gibba010 2021-06-23T12:34:58.253 INFO:teuthology.orchestra.console:Power on for gibba035 completed 2021-06-23T12:34:58.309 INFO:teuthology.orchestra.console:Power on for gibba010 completed 2021-06-23T12:34:58.355 INFO:teuthology.provision.fog.gibba035:Waiting for deploy to finish 2021-06-23T12:34:58.410 INFO:teuthology.provision.fog.gibba010:Waiting for deploy to finish 2021-06-23T12:38:59.101 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-23T12:39:09.176 INFO:teuthology.provision.fog.gibba010:Node is ready 2021-06-23T12:39:09.188 INFO:teuthology.orchestra.run.gibba010.stdout:gibba029 2021-06-23T12:39:09.240 INFO:teuthology.orchestra.run.gibba010.stdout:172.21.2.129 gibba029.front.sepia.ceph.com gibba029 2021-06-23T12:39:09.527 INFO:teuthology.provision.fog.gibba010:Deploy complete! 2021-06-23T12:39:46.550 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:47.646 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:48.740 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:49.835 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:50.928 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:52.023 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:53.116 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:54.210 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:55.304 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:39:56.359 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:02.415 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:03.471 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:04.565 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:05.659 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:06.662 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-06-23T12:40:06.663 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T12:40:06.663 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-06-23T12:40:06.664 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-06-23T12:40:06.664 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-06-23T12:40:06.664 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-06-23T12:40:06.665 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-06-23T12:40:06.665 ERROR:paramiko.transport: raise EOFError() 2021-06-23T12:40:06.665 ERROR:paramiko.transport:EOFError 2021-06-23T12:40:06.666 ERROR:paramiko.transport: 2021-06-23T12:40:06.666 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-06-23T12:40:06.666 ERROR:paramiko.transport: 2021-06-23T12:40:06.667 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T12:40:06.667 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-06-23T12:40:06.667 ERROR:paramiko.transport: self._check_banner() 2021-06-23T12:40:06.667 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-06-23T12:40:06.668 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-06-23T12:40:06.668 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-06-23T12:40:06.668 ERROR:paramiko.transport: 2021-06-23T12:40:12.764 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:13.859 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:14.953 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:16.047 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:17.142 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:18.198 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:19.292 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:20.386 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:21.481 ERROR:teuthology.orchestra.connection:Error authenticating with gibba035.front.sepia.ceph.com: Authentication failed. 2021-06-23T12:40:22.483 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-06-23T12:40:22.483 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T12:40:22.484 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-06-23T12:40:22.484 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-06-23T12:40:22.484 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-06-23T12:40:22.485 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-06-23T12:40:22.485 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-06-23T12:40:22.485 ERROR:paramiko.transport: raise EOFError() 2021-06-23T12:40:22.486 ERROR:paramiko.transport:EOFError 2021-06-23T12:40:22.486 ERROR:paramiko.transport: 2021-06-23T12:40:22.486 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-06-23T12:40:22.486 ERROR:paramiko.transport: 2021-06-23T12:40:22.487 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T12:40:22.487 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-06-23T12:40:22.488 ERROR:paramiko.transport: self._check_banner() 2021-06-23T12:40:22.488 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-06-23T12:40:22.488 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-06-23T12:40:22.489 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-06-23T12:40:22.489 ERROR:paramiko.transport: 2021-06-23T12:40:28.492 ERROR:paramiko.transport:Exception: Error reading SSH protocol banner 2021-06-23T12:40:28.493 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T12:40:28.493 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2211, in _check_banner 2021-06-23T12:40:28.493 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2021-06-23T12:40:28.494 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 380, in readline 2021-06-23T12:40:28.494 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2021-06-23T12:40:28.494 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 609, in _read_timeout 2021-06-23T12:40:28.495 ERROR:paramiko.transport: raise EOFError() 2021-06-23T12:40:28.495 ERROR:paramiko.transport:EOFError 2021-06-23T12:40:28.495 ERROR:paramiko.transport: 2021-06-23T12:40:28.496 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2021-06-23T12:40:28.496 ERROR:paramiko.transport: 2021-06-23T12:40:28.496 ERROR:paramiko.transport:Traceback (most recent call last): 2021-06-23T12:40:28.497 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2039, in run 2021-06-23T12:40:28.497 ERROR:paramiko.transport: self._check_banner() 2021-06-23T12:40:28.497 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2216, in _check_banner 2021-06-23T12:40:28.497 ERROR:paramiko.transport: "Error reading SSH protocol banner" + str(e) 2021-06-23T12:40:28.498 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2021-06-23T12:40:28.498 ERROR:paramiko.transport: 2021-06-23T12:42:55.811 INFO:teuthology.orchestra.run:Running command with timeout 600 2021-06-23T12:42:56.051 INFO:teuthology.provision.fog.gibba035:Node is ready 2021-06-23T12:42:56.063 INFO:teuthology.orchestra.run.gibba035.stdout:gibba035.front.sepia.ceph.com 2021-06-23T12:42:56.115 INFO:teuthology.orchestra.run.gibba035.stdout:172.21.2.135 gibba035.front.sepia.ceph.com gibba035 2021-06-23T12:42:56.410 INFO:teuthology.provision.fog.gibba035:Deploy complete! 2021-06-23T12:43:05.543 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan gibba010.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds 2021-06-23T12:43:05.591 INFO:teuthology.nuke:Checking targets against current locks 2021-06-23T12:43:05.681 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-23T12:43:05.685 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-06-23T12:43:05.739 INFO:teuthology.orchestra.console:Power off gibba010 2021-06-23T12:43:05.755 INFO:teuthology.orchestra.console:Power off gibba035 2021-06-23T12:43:05.905 INFO:teuthology.orchestra.console:Power off for gibba035 completed 2021-06-23T12:43:06.044 INFO:teuthology.lock.ops:unlocked gibba035.front.sepia.ceph.com 2021-06-23T12:43:14.118 INFO:teuthology.orchestra.console:Power off for gibba010 completed 2021-06-23T12:43:14.265 INFO:teuthology.lock.ops:unlocked gibba010.front.sepia.ceph.com 2021-06-23T12:43:14.266 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/virtualenv/bin/teuthology-dispatcher", line 33, in sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')()) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/scripts/dispatcher.py", line 33, in main teuthology.dispatcher.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/__init__.py", line 61, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 50, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/dispatcher/supervisor.py", line 163, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/lock/ops.py", line 318, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/lock/ops.py", line 285, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/misc.py", line 1103, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_f359b10daba6e0103d42ccfc021bc797f3cd7edc/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan gibba010.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds