2022-12-21T17:57:13.094 INFO:root:teuthology version: 0.0.1.dev18+g19d18db 2022-12-21T17:57:13.187 INFO:teuthology.lock.ops:Start node 'smithi074.front.sepia.ceph.com' reimaging 2022-12-21T17:57:13.188 INFO:teuthology.lock.ops:Updating [smithi074.front.sepia.ceph.com]: reset os type and version on server 2022-12-21T17:57:13.189 INFO:teuthology.lock.ops:Updating smithi074.front.sepia.ceph.com on lock server 2022-12-21T17:57:13.210 INFO:teuthology.lock.ops:Node 'smithi074.front.sepia.ceph.com' reimaging is complete 2022-12-21T17:57:13.372 INFO:teuthology.provision.fog.smithi074:Scheduling deploy of rhel 8.4 2022-12-21T17:57:13.633 INFO:teuthology.orchestra.console:Power off smithi074 2022-12-21T17:57:22.001 INFO:teuthology.orchestra.console:Power off for smithi074 completed 2022-12-21T17:57:22.103 INFO:teuthology.orchestra.console:Power on smithi074 2022-12-21T17:57:30.476 INFO:teuthology.orchestra.console:Power on for smithi074 completed 2022-12-21T17:57:30.577 INFO:teuthology.provision.fog.smithi074:Waiting for deploy to finish 2022-12-21T18:03:05.742 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:03:14.840 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:03:23.915 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:03:32.999 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:03:42.055 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:03:51.111 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:04:00.171 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:05:06.225 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:05:31.019 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:05:40.103 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:05:49.159 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:05:58.221 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:06:07.307 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:06:16.391 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:06:25.447 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:06:34.503 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:06:43.559 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:06:52.662 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:07:01.771 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:07:10.859 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:07:19.943 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:07:28.999 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:07:38.055 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:07:47.147 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:07:56.235 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:08:05.355 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:08:14.567 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:08:23.623 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:08:32.811 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:09:38.880 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:10:44.948 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:11:51.019 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:12:57.083 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:14:03.147 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:15:09.211 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:16:15.281 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:17:21.348 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:17:45.739 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:17:54.827 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:18:03.911 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:18:12.967 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:18:22.023 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:18:31.079 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:18:40.171 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:18:49.255 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:18:58.343 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:19:07.431 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:19:16.487 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:19:25.547 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:19:34.631 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:19:43.687 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:19:52.743 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:20:01.799 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:20:10.855 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:20:19.915 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:20:28.999 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:20:38.059 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:20:47.147 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:20:56.199 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:21:05.259 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:21:14.347 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:21:23.431 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:21:32.487 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:21:41.547 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:21:50.631 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:21:59.691 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:22:08.775 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:22:17.835 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:22:26.919 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:22:35.975 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:22:45.031 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:22:54.087 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:23:03.147 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:23:12.231 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:23:21.291 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:23:30.375 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:23:39.431 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:23:48.487 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:23:57.579 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:24:06.663 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:24:15.723 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:24:24.807 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:24:33.863 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:24:42.923 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:24:52.007 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:25:01.063 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:25:10.127 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:25:19.207 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:25:28.295 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:25:37.355 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:25:46.439 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:25:55.495 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:26:04.558 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:26:13.639 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.74 2022-12-21T18:27:19.704 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:28:25.773 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:29:31.839 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:30:37.904 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:31:43.980 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:32:50.044 WARNING:teuthology.provision.fog:timed out 2022-12-21T18:32:50.047 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/__init__.py", line 39, in reimage return obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 271, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds 2022-12-21T18:32:50.144 INFO:teuthology.nuke:Checking targets against current locks 2022-12-21T18:32:50.198 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-12-21T18:32:50.284 INFO:teuthology.orchestra.console:Power off smithi074 2022-12-21T18:32:54.558 INFO:teuthology.orchestra.console:Power off for smithi074 completed 2022-12-21T18:32:54.711 INFO:teuthology.lock.ops:unlocked: smithi074.front.sepia.ceph.com 2022-12-21T18:32:55.655 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/__init__.py", line 63, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 52, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage reimaged = reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 324, in reimage_machines log.info("Node '%s' reimaging is complete", machine) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/__init__.py", line 39, in reimage return obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 271, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/contextutil.py", line 133, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds