2023-06-05T20:32:21.593 INFO:root:teuthology version: 0.0.1.dev116+g62814ab 2023-06-05T20:32:21.810 INFO:teuthology.lock.ops:Start node 'smithi135.front.sepia.ceph.com' reimaging 2023-06-05T20:32:21.811 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: reset os type and version on server 2023-06-05T20:32:21.811 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2023-06-05T20:32:21.844 INFO:teuthology.lock.ops:Start node 'smithi156.front.sepia.ceph.com' reimaging 2023-06-05T20:32:21.844 INFO:teuthology.lock.ops:Updating [smithi156.front.sepia.ceph.com]: reset os type and version on server 2023-06-05T20:32:21.845 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-06-05T20:32:22.013 INFO:teuthology.provision.fog.smithi135:Scheduling deploy of rhel 8.6 2023-06-05T20:32:22.024 INFO:teuthology.provision.fog.smithi156:Scheduling deploy of rhel 8.6 2023-06-05T20:32:22.348 INFO:teuthology.orchestra.console:Power off smithi156 2023-06-05T20:32:22.362 INFO:teuthology.orchestra.console:Power off smithi135 2023-06-05T20:32:30.715 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2023-06-05T20:32:30.727 INFO:teuthology.orchestra.console:Power off for smithi135 completed 2023-06-05T20:32:30.816 INFO:teuthology.orchestra.console:Power on smithi156 2023-06-05T20:32:30.828 INFO:teuthology.orchestra.console:Power on smithi135 2023-06-05T20:32:35.078 INFO:teuthology.orchestra.console:Power on for smithi135 completed 2023-06-05T20:32:35.179 INFO:teuthology.provision.fog.smithi135:Waiting for deploy to finish 2023-06-05T20:32:39.182 INFO:teuthology.orchestra.console:Power on for smithi156 completed 2023-06-05T20:32:39.283 INFO:teuthology.provision.fog.smithi156:Waiting for deploy to finish 2023-06-05T20:39:07.376 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:39:16.464 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:39:25.520 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:39:34.576 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:39:43.632 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:39:52.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:40:01.744 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:40:10.801 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:40:19.856 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:40:28.912 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:40:37.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:40:47.024 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:40:56.080 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:41:05.136 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:41:14.192 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:41:23.248 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:41:32.304 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:41:41.360 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:41:50.416 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:41:59.476 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:42:08.528 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:42:17.584 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:42:26.640 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:42:35.696 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:42:44.752 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:42:53.808 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:43:02.864 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:43:11.920 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:43:20.976 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:43:30.032 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:43:39.088 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:43:48.144 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:43:57.200 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:44:06.288 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:44:15.344 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:44:24.400 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:44:33.456 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:44:42.512 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:44:51.568 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:45:00.624 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:45:09.680 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:45:18.736 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:45:27.792 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:45:36.848 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:45:45.908 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:45:54.992 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:46:04.048 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:46:13.104 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:46:22.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:46:31.216 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:46:40.272 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:46:49.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:46:58.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:47:07.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:47:16.528 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:47:25.584 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:47:34.640 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:47:43.696 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:47:52.755 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:48:01.841 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:48:10.896 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:48:19.984 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:48:29.040 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:48:38.128 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:48:47.189 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:48:56.272 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:49:05.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:49:14.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:49:23.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:49:32.496 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:49:41.552 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:49:50.608 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:49:59.664 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:50:08.720 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:50:17.776 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:50:26.832 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:50:35.920 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:50:44.976 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:50:54.032 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:51:03.088 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:51:12.144 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:51:21.200 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:51:30.256 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:51:39.312 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:51:48.368 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:51:57.424 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:52:06.480 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:52:15.536 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:52:24.592 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:52:33.648 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:52:42.704 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:52:51.760 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:53:00.816 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:53:09.872 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:53:18.928 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:53:27.984 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:53:37.040 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:53:46.096 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:53:55.152 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:54:04.208 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-06-05T20:54:04.210 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/dispatcher/supervisor.py", line 226, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/provision/fog.py", line 272, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2023-06-05T20:54:04.294 INFO:teuthology.nuke:Checking targets against current locks 2023-06-05T20:54:04.320 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/dispatcher/supervisor.py", line 226, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/provision/fog.py", line 272, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/dispatcher/supervisor.py", line 56, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/dispatcher/supervisor.py", line 234, in reimage nuke.nuke(ctx, True) File "/home/teuthworker/src/git.ceph.com_teuthology_62814aba234085f597115b9ba9bdc56af117de9e/teuthology/nuke/__init__.py", line 248, in nuke if ctx.name and ctx.name not in status.get('description', ""): TypeError: argument of type 'NoneType' is not iterable