2023-01-12T00:07:47.974 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2023-01-12T00:07:48.173 INFO:teuthology.lock.ops:Start node 'smithi026.front.sepia.ceph.com' reimaging 2023-01-12T00:07:48.174 INFO:teuthology.lock.ops:Updating [smithi026.front.sepia.ceph.com]: reset os type and version on server 2023-01-12T00:07:48.174 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2023-01-12T00:07:48.195 INFO:teuthology.lock.ops:Node 'smithi026.front.sepia.ceph.com' reimaging is complete 2023-01-12T00:07:48.195 INFO:teuthology.lock.ops:Start node 'smithi073.front.sepia.ceph.com' reimaging 2023-01-12T00:07:48.195 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: reset os type and version on server 2023-01-12T00:07:48.196 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2023-01-12T00:07:52.801 INFO:teuthology.lock.ops:Node 'smithi073.front.sepia.ceph.com' reimaging is complete 2023-01-12T00:07:52.802 INFO:teuthology.lock.ops:Start node 'smithi123.front.sepia.ceph.com' reimaging 2023-01-12T00:07:52.802 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: reset os type and version on server 2023-01-12T00:07:52.803 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-01-12T00:08:17.826 INFO:teuthology.lock.ops:Node 'smithi123.front.sepia.ceph.com' reimaging is complete 2023-01-12T00:08:17.826 INFO:teuthology.lock.ops:Start node 'smithi173.front.sepia.ceph.com' reimaging 2023-01-12T00:08:17.827 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: reset os type and version on server 2023-01-12T00:08:17.827 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2023-01-12T00:08:23.520 INFO:teuthology.lock.ops:Node 'smithi173.front.sepia.ceph.com' reimaging is complete 2023-01-12T00:08:23.636 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of rhel 8.6 2023-01-12T00:08:23.641 INFO:teuthology.provision.fog.smithi026:Scheduling deploy of rhel 8.6 2023-01-12T00:08:23.667 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of rhel 8.6 2023-01-12T00:08:23.696 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of rhel 8.6 2023-01-12T00:08:23.960 INFO:teuthology.orchestra.console:Power off smithi026 2023-01-12T00:08:23.977 INFO:teuthology.orchestra.console:Power off smithi123 2023-01-12T00:08:23.992 INFO:teuthology.orchestra.console:Power off smithi173 2023-01-12T00:08:24.005 INFO:teuthology.orchestra.console:Power off smithi073 2023-01-12T00:08:37.493 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2023-01-12T00:08:37.512 INFO:teuthology.orchestra.console:Power off for smithi073 completed 2023-01-12T00:08:37.594 INFO:teuthology.orchestra.console:Power on smithi173 2023-01-12T00:08:37.613 INFO:teuthology.orchestra.console:Power on smithi073 2023-01-12T00:08:42.594 INFO:teuthology.orchestra.console:Power off for smithi123 completed 2023-01-12T00:08:42.613 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2023-01-12T00:08:42.696 INFO:teuthology.orchestra.console:Power on smithi123 2023-01-12T00:08:42.714 INFO:teuthology.orchestra.console:Power on smithi026 2023-01-12T00:08:46.957 INFO:teuthology.orchestra.console:Power on for smithi123 completed 2023-01-12T00:08:46.986 INFO:teuthology.orchestra.console:Power on for smithi073 completed 2023-01-12T00:08:47.059 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish 2023-01-12T00:08:47.087 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish 2023-01-12T00:08:51.091 INFO:teuthology.orchestra.console:Power on for smithi026 completed 2023-01-12T00:08:51.192 INFO:teuthology.provision.fog.smithi026:Waiting for deploy to finish 2023-01-12T00:08:59.545 INFO:teuthology.orchestra.console:Power on for smithi173 completed 2023-01-12T00:08:59.646 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2023-01-12T00:11:33.708 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-01-12T00:11:47.788 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-01-12T00:11:49.324 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-01-12T00:11:49.325 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-01-12T00:11:58.380 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-01-12T00:11:58.381 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-01-12T00:12:07.436 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-01-12T00:12:07.437 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-01-12T00:12:23.447 ERROR:teuthology.orchestra.connection:Error authenticating with smithi073.front.sepia.ceph.com: Authentication timeout. 2023-01-12T00:12:41.804 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.173 2023-01-12T00:12:53.305 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-12T00:12:53.572 INFO:teuthology.provision.fog.smithi173:Node is ready 2023-01-12T00:12:53.594 INFO:teuthology.orchestra.run.smithi173.stdout:smithi022 2023-01-12T00:12:53.650 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2023-01-12T00:12:53.792 WARNING:teuthology.provision.fog:timed out 2023-01-12T00:12:53.994 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2023-01-12T00:13:00.328 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-01-12T00:13:00.639 INFO:teuthology.provision.fog.smithi026:Node is ready 2023-01-12T00:13:00.658 INFO:teuthology.orchestra.run.smithi026.stdout:smithi026.front.sepia.ceph.com 2023-01-12T00:13:00.717 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26 smithi026.front.sepia.ceph.com smithi026 2023-01-12T00:13:01.071 INFO:teuthology.provision.fog.smithi026:Deploy complete! 2023-01-12T00:13:01.075 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/__init__.py", line 39, in reimage return obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 273, in _wait_for_ready self.remote.connect() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/orchestra/remote.py", line 386, in connect self.ssh = connection.connect(**args) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/orchestra/connection.py", line 105, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 420, in connect if our_key != server_key: File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/pkey.py", line 143, in __eq__ return self._fields == other._fields AttributeError: 'NoneType' object has no attribute '_fields' 2023-01-12T00:13:03.122 INFO:teuthology.nuke:Checking targets against current locks 2023-01-12T00:13:03.171 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-12T00:13:03.198 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-12T00:13:03.230 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-12T00:13:03.264 INFO:teuthology.orchestra.console:Power off smithi026 2023-01-12T00:13:03.281 INFO:teuthology.orchestra.console:Power off smithi073 2023-01-12T00:13:03.290 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-12T00:13:03.357 INFO:teuthology.orchestra.console:Power off smithi123 2023-01-12T00:13:03.397 INFO:teuthology.orchestra.console:Power off smithi173 2023-01-12T00:13:11.650 INFO:teuthology.orchestra.console:Power off for smithi026 completed 2023-01-12T00:13:11.657 INFO:teuthology.orchestra.console:Power off for smithi073 completed 2023-01-12T00:13:11.773 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2023-01-12T00:13:11.796 INFO:teuthology.lock.ops:unlocked: smithi026.front.sepia.ceph.com 2023-01-12T00:13:11.800 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com 2023-01-12T00:13:11.917 INFO:teuthology.lock.ops:unlocked: smithi173.front.sepia.ceph.com 2023-01-12T00:13:18.441 WARNING:teuthology.provision.fog:timed out 2023-01-12T00:13:21.596 INFO:teuthology.orchestra.console:Power off for smithi123 completed 2023-01-12T00:13:24.605 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com 2023-01-12T00:13:42.988 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/__init__.py", line 63, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 52, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/__init__.py", line 39, in reimage return obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 273, in _wait_for_ready self.remote.connect() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/orchestra/remote.py", line 386, in connect self.ssh = connection.connect(**args) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/orchestra/connection.py", line 105, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 420, in connect if our_key != server_key: File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/lib/python3.8/site-packages/paramiko/pkey.py", line 143, in __eq__ return self._fields == other._fields AttributeError: 'NoneType' object has no attribute '_fields'