2023-01-24T08:46:42.290 INFO:root:teuthology version: 0.0.1.dev20+g5abecab
2023-01-24T08:46:42.401 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging
2023-01-24T08:46:42.403 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server
2023-01-24T08:46:42.403 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server
2023-01-24T08:46:42.421 INFO:teuthology.lock.ops:Node 'smithi072.front.sepia.ceph.com' reimaging is complete
2023-01-24T08:46:42.422 INFO:teuthology.lock.ops:Start node 'smithi085.front.sepia.ceph.com' reimaging
2023-01-24T08:46:42.422 INFO:teuthology.lock.ops:Updating [smithi085.front.sepia.ceph.com]: reset os type and version on server
2023-01-24T08:46:42.423 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server
2023-01-24T08:46:42.437 INFO:teuthology.lock.ops:Node 'smithi085.front.sepia.ceph.com' reimaging is complete
2023-01-24T08:46:42.577 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of rhel 8.6
2023-01-24T08:46:42.585 INFO:teuthology.provision.fog.smithi085:Scheduling deploy of rhel 8.6
2023-01-24T08:46:42.806 INFO:teuthology.orchestra.console:Power off smithi072
2023-01-24T08:46:42.820 INFO:teuthology.orchestra.console:Power off smithi085
2023-01-24T08:46:51.174 INFO:teuthology.orchestra.console:Power off for smithi072 completed
2023-01-24T08:46:51.183 INFO:teuthology.orchestra.console:Power off for smithi085 completed
2023-01-24T08:46:51.276 INFO:teuthology.orchestra.console:Power on smithi072
2023-01-24T08:46:51.284 INFO:teuthology.orchestra.console:Power on smithi085
2023-01-24T08:46:59.637 INFO:teuthology.orchestra.console:Power on for smithi072 completed
2023-01-24T08:46:59.642 INFO:teuthology.orchestra.console:Power on for smithi085 completed
2023-01-24T08:46:59.739 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish
2023-01-24T08:46:59.743 INFO:teuthology.provision.fog.smithi085:Waiting for deploy to finish
2023-01-24T08:49:49.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T08:49:58.216 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T08:50:07.272 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T08:50:16.360 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T08:50:25.416 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T08:50:34.216 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:50:43.276 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:50:52.364 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:51:01.448 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:51:03.240 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-24T08:51:03.594 INFO:teuthology.provision.fog.smithi085:Node is ready
2023-01-24T08:51:03.609 INFO:teuthology.orchestra.run.smithi085.stdout:smithi022
2023-01-24T08:51:03.667 INFO:teuthology.orchestra.run.smithi085.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-01-24T08:51:04.195 INFO:teuthology.provision.fog.smithi085:Deploy complete!
2023-01-24T08:51:10.508 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:52:16.574 WARNING:teuthology.provision.fog:timed out
2023-01-24T08:52:40.940 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:52:50.024 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:52:59.087 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:53:08.172 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:53:17.256 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:53:26.316 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:53:35.400 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:53:44.488 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:53:53.544 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:54:02.632 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:54:11.692 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:54:20.776 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:54:29.836 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:54:38.924 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:54:48.008 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:54:57.064 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:55:06.120 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:55:15.208 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:55:24.264 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:55:33.320 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:55:42.376 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:55:51.432 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:56:00.488 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:56:09.544 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:56:18.600 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:56:27.660 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:56:36.744 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:56:45.804 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:56:54.888 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:57:03.944 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:57:13.032 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:57:22.120 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:57:31.176 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:57:40.232 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:57:49.288 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:57:58.348 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:58:07.432 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:58:16.488 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:58:25.576 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:58:34.632 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:58:43.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:58:52.744 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:59:01.832 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:59:10.892 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:59:19.976 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:59:29.036 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:59:38.120 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:59:47.180 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T08:59:56.268 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:00:05.352 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:00:14.408 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:00:23.464 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:00:32.524 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:00:41.612 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:00:50.700 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:00:59.752 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:01:08.812 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:01:17.900 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:01:26.984 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:01:36.040 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:01:45.096 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:01:54.156 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:02:03.244 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:02:12.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:02:21.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:02:30.444 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:02:39.532 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:02:48.620 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:02:57.704 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:03:06.764 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:03:15.852 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:03:24.936 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:03:33.992 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:03:43.048 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:03:52.104 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:04:01.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:04:10.220 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:04:19.308 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:04:28.392 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:04:37.448 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:04:46.508 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:04:55.592 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:05:04.648 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:05:13.704 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:05:22.792 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:05:31.852 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:05:40.936 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:05:49.992 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:05:59.048 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:06:08.108 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:06:17.192 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:06:26.248 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:06:35.336 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:06:44.392 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T09:06:44.394 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/provision/fog.py", line 271, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds
2023-01-24T09:06:44.448 INFO:teuthology.nuke:Checking targets against current locks
2023-01-24T09:06:44.495 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-24T09:06:44.516 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-24T09:06:44.566 INFO:teuthology.orchestra.console:Power off smithi072
2023-01-24T09:06:44.599 INFO:teuthology.orchestra.console:Power off smithi085
2023-01-24T09:06:52.955 INFO:teuthology.orchestra.console:Power off for smithi085 completed
2023-01-24T09:06:53.105 INFO:teuthology.lock.ops:unlocked: smithi085.front.sepia.ceph.com
2023-01-24T09:06:57.045 INFO:teuthology.orchestra.console:Power off for smithi072 completed
2023-01-24T09:06:57.200 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com
2023-01-24T09:06:57.410 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/dispatcher/__init__.py", line 63, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/dispatcher/supervisor.py", line 52, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/provision/fog.py", line 271, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_5abecab36c9da4d70461d81781464cf5acaea292/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds