2022-12-15T06:43:04.167 INFO:root:teuthology version: 0.0.1.dev18+g19d18db
2022-12-15T06:43:29.316 INFO:teuthology.lock.ops:Start node 'smithi094.front.sepia.ceph.com' reimaging
2022-12-15T06:43:29.316 INFO:teuthology.lock.ops:Updating [smithi094.front.sepia.ceph.com]: reset os type and version on server
2022-12-15T06:43:29.316 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server
2022-12-15T06:43:34.343 INFO:teuthology.lock.ops:Node 'smithi094.front.sepia.ceph.com' reimaging is complete
2022-12-15T06:43:34.344 INFO:teuthology.lock.ops:Start node 'smithi106.front.sepia.ceph.com' reimaging
2022-12-15T06:43:34.344 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: reset os type and version on server
2022-12-15T06:43:34.344 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server
2022-12-15T06:43:39.368 INFO:teuthology.lock.ops:Node 'smithi106.front.sepia.ceph.com' reimaging is complete
2022-12-15T06:43:49.594 INFO:teuthology.provision.fog.smithi094:Scheduling deploy of centos 8.stream
2022-12-15T06:43:49.597 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of centos 8.stream
2022-12-15T06:43:57.771 INFO:teuthology.orchestra.console:Power off smithi106
2022-12-15T06:43:57.791 INFO:teuthology.orchestra.console:Power off smithi094
2022-12-15T06:44:07.170 INFO:teuthology.orchestra.console:Power off for smithi094 completed
2022-12-15T06:44:07.271 INFO:teuthology.orchestra.console:Power on smithi094
2022-12-15T06:44:17.385 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2022-12-15T06:44:17.486 INFO:teuthology.orchestra.console:Power on smithi106
2022-12-15T06:44:26.862 INFO:teuthology.orchestra.console:Power on for smithi106 completed
2022-12-15T06:44:26.963 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish
2022-12-15T06:45:16.336 INFO:teuthology.orchestra.console:Power on for smithi094 completed
2022-12-15T06:45:16.437 INFO:teuthology.provision.fog.smithi094:Waiting for deploy to finish
2022-12-15T06:45:34.912 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.94
2022-12-15T06:45:50.559 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.94
2022-12-15T06:46:04.643 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.94
2022-12-15T06:46:14.431 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.94
2022-12-15T06:46:23.541 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:29.603 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:30.663 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:31.762 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:37.867 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:38.964 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:45.068 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:51.170 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:55.424 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:46:55.672 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-12-15T06:46:56.018 INFO:teuthology.provision.fog.smithi094:Node is ready
2022-12-15T06:46:56.042 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094
2022-12-15T06:46:56.101 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94 smithi094 smithi094
2022-12-15T06:47:01.468 INFO:teuthology.provision.fog.smithi094:Deploy complete!
2022-12-15T06:47:01.525 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:47:01.525 WARNING:teuthology.provision.fog:'connect to smithi106.front.sepia.ceph.com' reached maximum tries (10) after waiting for 10 seconds
2022-12-15T06:47:07.627 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:47:13.732 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:47:19.738 ERROR:paramiko.transport:Exception (client): Error reading SSH protocol banner
2022-12-15T06:47:19.740 ERROR:paramiko.transport:Traceback (most recent call last):
2022-12-15T06:47:19.741 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2270, in _check_banner
2022-12-15T06:47:19.741 ERROR:paramiko.transport:    buf = self.packetizer.readline(timeout)
2022-12-15T06:47:19.741 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 380, in readline
2022-12-15T06:47:19.742 ERROR:paramiko.transport:    buf += self._read_timeout(timeout)
2022-12-15T06:47:19.742 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 609, in _read_timeout
2022-12-15T06:47:19.743 ERROR:paramiko.transport:    raise EOFError()
2022-12-15T06:47:19.743 ERROR:paramiko.transport:EOFError
2022-12-15T06:47:19.743 ERROR:paramiko.transport:
2022-12-15T06:47:19.743 ERROR:paramiko.transport:During handling of the above exception, another exception occurred:
2022-12-15T06:47:19.743 ERROR:paramiko.transport:
2022-12-15T06:47:19.743 ERROR:paramiko.transport:Traceback (most recent call last):
2022-12-15T06:47:19.744 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2093, in run
2022-12-15T06:47:19.744 ERROR:paramiko.transport:    self._check_banner()
2022-12-15T06:47:19.744 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2274, in _check_banner
2022-12-15T06:47:19.744 ERROR:paramiko.transport:    raise SSHException(
2022-12-15T06:47:19.744 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
2022-12-15T06:47:19.744 ERROR:paramiko.transport:
2022-12-15T06:47:19.745 WARNING:teuthology.provision.fog:Error reading SSH protocol banner
2022-12-15T06:47:25.844 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:47:26.943 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:47:32.949 ERROR:paramiko.transport:Exception (client): Error reading SSH protocol banner
2022-12-15T06:47:32.950 ERROR:paramiko.transport:Traceback (most recent call last):
2022-12-15T06:47:32.950 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2270, in _check_banner
2022-12-15T06:47:32.950 ERROR:paramiko.transport:    buf = self.packetizer.readline(timeout)
2022-12-15T06:47:32.950 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 380, in readline
2022-12-15T06:47:32.950 ERROR:paramiko.transport:    buf += self._read_timeout(timeout)
2022-12-15T06:47:32.950 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 609, in _read_timeout
2022-12-15T06:47:32.950 ERROR:paramiko.transport:    raise EOFError()
2022-12-15T06:47:32.950 ERROR:paramiko.transport:EOFError
2022-12-15T06:47:32.950 ERROR:paramiko.transport:
2022-12-15T06:47:32.951 ERROR:paramiko.transport:During handling of the above exception, another exception occurred:
2022-12-15T06:47:32.951 ERROR:paramiko.transport:
2022-12-15T06:47:32.951 ERROR:paramiko.transport:Traceback (most recent call last):
2022-12-15T06:47:32.951 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2093, in run
2022-12-15T06:47:32.951 ERROR:paramiko.transport:    self._check_banner()
2022-12-15T06:47:32.951 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2274, in _check_banner
2022-12-15T06:47:32.951 ERROR:paramiko.transport:    raise SSHException(
2022-12-15T06:47:32.951 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
2022-12-15T06:47:32.951 ERROR:paramiko.transport:
2022-12-15T06:47:32.951 WARNING:teuthology.provision.fog:Error reading SSH protocol banner
2022-12-15T06:47:39.052 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:47:40.055 ERROR:paramiko.transport:Exception (client): Error reading SSH protocol banner
2022-12-15T06:47:40.056 ERROR:paramiko.transport:Traceback (most recent call last):
2022-12-15T06:47:40.056 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2270, in _check_banner
2022-12-15T06:47:40.056 ERROR:paramiko.transport:    buf = self.packetizer.readline(timeout)
2022-12-15T06:47:40.056 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 380, in readline
2022-12-15T06:47:40.056 ERROR:paramiko.transport:    buf += self._read_timeout(timeout)
2022-12-15T06:47:40.056 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 609, in _read_timeout
2022-12-15T06:47:40.056 ERROR:paramiko.transport:    raise EOFError()
2022-12-15T06:47:40.056 ERROR:paramiko.transport:EOFError
2022-12-15T06:47:40.056 ERROR:paramiko.transport:
2022-12-15T06:47:40.056 ERROR:paramiko.transport:During handling of the above exception, another exception occurred:
2022-12-15T06:47:40.056 ERROR:paramiko.transport:
2022-12-15T06:47:40.057 ERROR:paramiko.transport:Traceback (most recent call last):
2022-12-15T06:47:40.057 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2093, in run
2022-12-15T06:47:40.057 ERROR:paramiko.transport:    self._check_banner()
2022-12-15T06:47:40.057 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2274, in _check_banner
2022-12-15T06:47:40.057 ERROR:paramiko.transport:    raise SSHException(
2022-12-15T06:47:40.057 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
2022-12-15T06:47:40.057 ERROR:paramiko.transport:
2022-12-15T06:47:40.057 WARNING:teuthology.provision.fog:Error reading SSH protocol banner
2022-12-15T06:47:51.162 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:47:56.344 ERROR:teuthology.orchestra.connection:Error authenticating with smithi106.front.sepia.ceph.com: Authentication failed.
2022-12-15T06:49:02.353 WARNING:teuthology.provision.fog:timed out
2022-12-15T06:49:26.847 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.106
2022-12-15T06:49:38.478 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-12-15T06:50:03.460 INFO:teuthology.provision.fog.smithi106:Node is ready
2022-12-15T06:50:03.485 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com
2022-12-15T06:50:03.547 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106
2022-12-15T06:50:03.943 INFO:teuthology.provision.fog.smithi106:Deploy complete!
2022-12-15T06:50:20.137 ERROR:teuthology.misc:write (smithi094.front.sepia.ceph.com): No route to host
2022-12-15T06:50:21.280 ERROR:teuthology.misc:write (smithi094.front.sepia.ceph.com): No route to host
2022-12-15T06:50:21.280 ERROR:teuthology.misc:write (smithi094.front.sepia.ceph.com): No route to host
2022-12-15T06:50:21.280 ERROR:teuthology.misc:write (smithi094.front.sepia.ceph.com): No route to host
2022-12-15T06:50:33.296 ERROR:teuthology.misc:write (smithi094.front.sepia.ceph.com): No route to host
2022-12-15T06:50:55.322 ERROR:teuthology.misc:write (smithi094.front.sepia.ceph.com): No route to host
2022-12-15T06:50:57.332 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 288, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/misc.py", line 1106, in ssh_keyscan
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi094.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds
2022-12-15T06:51:02.399 INFO:teuthology.nuke:Checking targets against current locks
2022-12-15T06:51:19.629 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-15T06:51:23.195 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-15T06:51:23.292 INFO:teuthology.orchestra.console:Power off smithi094
2022-12-15T06:51:23.320 INFO:teuthology.orchestra.console:Power off smithi106
2022-12-15T06:52:07.499 INFO:teuthology.orchestra.console:Power off for smithi094 completed
2022-12-15T06:52:15.898 INFO:teuthology.orchestra.console:Power off for smithi106 completed
2022-12-15T06:52:15.924 INFO:teuthology.lock.ops:unlocked: smithi094.front.sepia.ceph.com
2022-12-15T06:52:16.055 INFO:teuthology.lock.ops:unlocked: smithi106.front.sepia.ceph.com
2022-12-15T06:52:16.310 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/__init__.py", line 63, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 52, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 288, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/misc.py", line 1106, in ssh_keyscan
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi094.front.sepia.ceph.com' reached maximum tries (5) after waiting for 5 seconds