2023-04-10T18:45:05.328 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-04-10T18:45:05.441 INFO:teuthology.lock.ops:Start node 'smithi178.front.sepia.ceph.com' reimaging 2023-04-10T18:45:05.442 INFO:teuthology.lock.ops:Updating [smithi178.front.sepia.ceph.com]: reset os type and version on server 2023-04-10T18:45:05.443 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-04-10T18:45:05.468 INFO:teuthology.lock.ops:Node 'smithi178.front.sepia.ceph.com' reimaging is complete 2023-04-10T18:45:05.618 INFO:teuthology.provision.fog.smithi178:Scheduling deploy of centos 8.stream 2023-04-10T18:45:05.993 INFO:teuthology.orchestra.console:Power off smithi178 2023-04-10T18:45:14.362 INFO:teuthology.orchestra.console:Power off for smithi178 completed 2023-04-10T18:45:14.463 INFO:teuthology.orchestra.console:Power on smithi178 2023-04-10T18:45:22.829 INFO:teuthology.orchestra.console:Power on for smithi178 completed 2023-04-10T18:45:22.931 INFO:teuthology.provision.fog.smithi178:Waiting for deploy to finish 2023-04-10T18:48:59.653 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:49:08.741 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:49:17.829 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:49:26.885 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:49:35.941 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:50:42.006 WARNING:teuthology.provision.fog:timed out 2023-04-10T18:51:06.373 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:51:15.461 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:51:24.549 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:51:33.605 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:51:42.693 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:51:51.749 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:52:00.805 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:52:09.897 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:52:18.981 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:52:28.069 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:52:37.125 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:52:46.213 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:52:55.301 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:53:04.389 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:53:13.445 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:53:22.501 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:53:31.589 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:53:40.645 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:53:49.701 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:53:58.757 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:54:07.813 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:54:16.869 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:54:25.925 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:54:34.981 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:54:44.038 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:54:53.093 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:55:02.149 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:55:11.237 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:55:20.293 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:55:29.349 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:55:38.437 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:55:47.493 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:55:56.549 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:56:05.605 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:56:14.697 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:56:23.781 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:56:32.837 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:56:41.893 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:56:50.949 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:57:00.037 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:57:09.093 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:57:18.149 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:57:27.237 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:57:36.293 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:57:45.349 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:57:54.405 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:58:03.493 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:58:12.549 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:58:21.605 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:58:30.661 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:58:39.717 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:58:48.773 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:58:57.829 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:59:06.885 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:59:15.941 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:59:24.997 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:59:34.085 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:59:43.173 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T18:59:52.261 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:00:01.317 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:00:10.405 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:00:19.461 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:00:28.549 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:00:37.637 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:00:46.725 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:00:55.813 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:01:04.869 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:01:13.925 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:01:23.013 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:01:32.101 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:01:41.157 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:01:50.245 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:01:59.301 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:02:08.357 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:02:17.445 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:02:26.501 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:02:35.557 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:02:44.613 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:02:53.701 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:03:02.789 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:03:11.845 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:03:20.933 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:03:29.989 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:03:39.045 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:03:48.101 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:03:57.157 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:04:06.245 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:04:15.301 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:04:24.357 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:04:33.445 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:04:42.533 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:04:51.621 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:05:00.677 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:05:09.733 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-04-10T19:05:09.735 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/supervisor.py", line 226, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/__init__.py", line 39, in reimage return obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/fog.py", line 271, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/contextutil.py", line 135, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds 2023-04-10T19:05:09.823 INFO:teuthology.nuke:Checking targets against current locks 2023-04-10T19:05:09.847 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/supervisor.py", line 226, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/__init__.py", line 39, in reimage return obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/fog.py", line 271, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/contextutil.py", line 135, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/supervisor.py", line 56, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/supervisor.py", line 234, in reimage nuke.nuke(ctx, True) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/nuke/__init__.py", line 248, in nuke if ctx.name and ctx.name not in status.get('description', ""): TypeError: argument of type 'NoneType' is not iterable