2023-04-12T06:24:02.700 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-04-12T06:24:02.845 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging 2023-04-12T06:24:02.846 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server 2023-04-12T06:24:02.847 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2023-04-12T06:24:02.871 INFO:teuthology.lock.ops:Node 'smithi033.front.sepia.ceph.com' reimaging is complete 2023-04-12T06:24:02.872 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging 2023-04-12T06:24:02.872 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server 2023-04-12T06:24:02.873 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2023-04-12T06:24:02.895 INFO:teuthology.lock.ops:Node 'smithi191.front.sepia.ceph.com' reimaging is complete 2023-04-12T06:24:03.015 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of rhel 8.6 2023-04-12T06:24:03.035 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of rhel 8.6 2023-04-12T06:24:03.298 INFO:teuthology.orchestra.console:Power off smithi033 2023-04-12T06:24:03.311 INFO:teuthology.orchestra.console:Power off smithi191 2023-04-12T06:24:03.447 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2023-04-12T06:24:03.548 INFO:teuthology.orchestra.console:Power on smithi033 2023-04-12T06:24:07.799 INFO:teuthology.orchestra.console:Power on for smithi033 completed 2023-04-12T06:24:07.900 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2023-04-12T06:24:11.684 INFO:teuthology.orchestra.console:Power off for smithi191 completed 2023-04-12T06:24:11.785 INFO:teuthology.orchestra.console:Power on smithi191 2023-04-12T06:24:20.152 INFO:teuthology.orchestra.console:Power on for smithi191 completed 2023-04-12T06:24:20.253 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish 2023-04-12T06:26:42.245 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:26:51.333 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:27:00.421 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:27:09.477 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:27:18.533 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:27:27.589 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:27:36.645 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:27:45.701 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:27:54.757 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:28:03.813 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:28:12.869 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:28:21.925 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:28:31.013 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:28:40.069 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:28:49.129 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:28:58.213 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:29:07.301 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:29:16.357 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:29:25.413 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:29:34.469 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:29:43.525 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:29:52.581 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:30:01.669 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:30:10.757 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:30:19.813 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:30:28.869 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:30:37.925 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:30:46.981 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:30:56.037 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:31:05.093 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:31:14.149 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:31:23.209 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:31:32.293 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:31:41.349 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:31:50.405 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:31:59.461 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:32:08.517 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:32:17.605 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:32:26.661 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:32:35.717 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:32:44.773 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:32:53.829 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:33:02.885 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:33:11.941 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:33:21.033 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:33:30.117 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:33:39.177 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:33:48.261 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:33:57.317 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:34:06.377 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:34:15.461 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:34:24.517 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:34:33.573 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:34:42.633 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:34:51.685 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:35:00.741 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:35:09.797 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:35:18.853 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:35:27.909 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:35:36.965 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:35:46.021 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:35:55.077 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:36:04.133 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:36:13.189 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:36:22.245 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:36:31.301 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:36:40.361 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:36:49.445 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:36:58.501 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:37:07.560 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:37:16.645 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:37:25.704 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:37:34.789 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:37:43.849 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:37:52.937 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:38:02.021 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:38:11.077 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:38:20.133 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:38:29.193 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:38:38.285 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:38:47.365 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:38:56.421 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:39:05.477 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-04-12T06:39:10.334 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/supervisor.py", line 226, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/__init__.py", line 39, in reimage return obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/fog.py", line 86, in create self.wait_for_deploy_task(task_id) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/fog.py", line 255, in wait_for_deploy_task while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/contextutil.py", line 135, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (60) after waiting for 900 seconds 2023-04-12T06:39:10.403 INFO:teuthology.nuke:Checking targets against current locks 2023-04-12T06:39:10.427 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/supervisor.py", line 226, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/__init__.py", line 39, in reimage return obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/fog.py", line 86, in create self.wait_for_deploy_task(task_id) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/provision/fog.py", line 255, in wait_for_deploy_task while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/contextutil.py", line 135, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (60) after waiting for 900 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/supervisor.py", line 56, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/dispatcher/supervisor.py", line 234, in reimage nuke.nuke(ctx, True) File "/home/teuthworker/src/git.ceph.com_teuthology_8d156aede5efdae00b53d8d3b8d127082980e7ec/teuthology/nuke/__init__.py", line 248, in nuke if ctx.name and ctx.name not in status.get('description', ""): TypeError: argument of type 'NoneType' is not iterable