2023-01-24T12:22:40.097 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1
2023-01-24T12:22:40.228 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging
2023-01-24T12:22:40.229 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server
2023-01-24T12:22:40.229 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server
2023-01-24T12:22:40.246 INFO:teuthology.lock.ops:Node 'smithi072.front.sepia.ceph.com' reimaging is complete
2023-01-24T12:22:40.247 INFO:teuthology.lock.ops:Start node 'smithi085.front.sepia.ceph.com' reimaging
2023-01-24T12:22:40.247 INFO:teuthology.lock.ops:Updating [smithi085.front.sepia.ceph.com]: reset os type and version on server
2023-01-24T12:22:40.247 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server
2023-01-24T12:22:40.262 INFO:teuthology.lock.ops:Node 'smithi085.front.sepia.ceph.com' reimaging is complete
2023-01-24T12:22:40.403 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of rhel 8.6
2023-01-24T12:22:40.416 INFO:teuthology.provision.fog.smithi085:Scheduling deploy of rhel 8.6
2023-01-24T12:22:40.643 INFO:teuthology.orchestra.console:Power off smithi072
2023-01-24T12:22:40.660 INFO:teuthology.orchestra.console:Power off smithi085
2023-01-24T12:22:40.789 INFO:teuthology.orchestra.console:Power off for smithi072 completed
2023-01-24T12:22:40.801 INFO:teuthology.orchestra.console:Power off for smithi085 completed
2023-01-24T12:22:40.891 INFO:teuthology.orchestra.console:Power on smithi072
2023-01-24T12:22:40.901 INFO:teuthology.orchestra.console:Power on smithi085
2023-01-24T12:22:45.146 INFO:teuthology.orchestra.console:Power on for smithi072 completed
2023-01-24T12:22:45.151 INFO:teuthology.orchestra.console:Power on for smithi085 completed
2023-01-24T12:22:45.247 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish
2023-01-24T12:22:45.252 INFO:teuthology.provision.fog.smithi085:Waiting for deploy to finish
2023-01-24T12:25:34.568 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T12:25:43.628 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T12:25:52.716 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T12:26:01.800 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T12:26:10.860 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2023-01-24T12:26:19.884 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:26:28.972 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:26:38.056 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:26:47.112 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:27:16.888 WARNING:teuthology.provision.fog:timed out
2023-01-24T12:27:23.221 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-24T12:27:23.475 INFO:teuthology.provision.fog.smithi085:Node is ready
2023-01-24T12:27:23.533 INFO:teuthology.orchestra.run.smithi085.stdout:smithi085
2023-01-24T12:27:23.591 INFO:teuthology.orchestra.run.smithi085.stdout:172.21.15.85 smithi085.front.sepia.ceph.com smithi085
2023-01-24T12:27:23.929 INFO:teuthology.provision.fog.smithi085:Deploy complete!
2023-01-24T12:27:53.145 WARNING:teuthology.provision.fog:timed out
2023-01-24T12:28:17.640 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:28:26.696 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:28:35.756 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:28:44.840 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:28:53.900 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:29:02.988 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:29:12.076 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:29:21.129 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:29:30.184 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:29:39.244 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:29:48.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:29:57.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:30:06.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:30:15.500 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:30:24.584 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:30:33.640 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:30:42.696 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:30:51.756 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:31:00.844 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:31:09.928 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:31:18.988 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:31:28.076 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:31:37.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:31:46.216 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:31:55.272 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:32:04.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:32:13.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:32:22.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:32:31.496 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:32:40.556 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:32:49.644 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:32:58.732 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:33:07.816 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:33:16.876 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:33:25.964 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:33:35.048 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:33:44.104 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:33:53.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:34:02.216 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:34:11.272 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:34:20.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:34:29.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:34:38.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:34:47.496 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:34:56.552 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:35:05.640 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:35:14.700 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:35:23.784 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:35:32.844 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:35:41.928 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:35:50.984 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:36:00.040 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:36:09.096 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:36:18.152 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:36:27.212 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:36:36.300 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:36:45.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:36:54.444 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:37:03.528 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:37:12.584 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:37:21.644 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:37:30.728 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:37:39.788 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:37:48.876 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:37:57.960 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:38:07.016 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:38:16.076 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:38:25.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:38:34.220 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:38:43.308 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:38:52.392 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:39:01.452 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:39:10.536 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:39:19.593 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:39:28.652 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:39:37.736 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:39:46.796 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:39:55.880 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:40:04.940 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:40:14.024 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:40:23.080 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:40:32.136 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:40:41.192 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:40:50.252 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:40:59.340 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:41:08.424 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:41:17.484 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:41:26.568 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:41:35.628 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:41:44.716 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:41:53.804 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:42:02.888 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:42:11.944 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:42:21.004 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:42:30.088 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2023-01-24T12:42:30.090 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/fog.py", line 271, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds
2023-01-24T12:42:30.148 INFO:teuthology.nuke:Checking targets against current locks
2023-01-24T12:42:30.196 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-24T12:42:30.218 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-24T12:42:30.268 INFO:teuthology.orchestra.console:Power off smithi072
2023-01-24T12:42:30.301 INFO:teuthology.orchestra.console:Power off smithi085
2023-01-24T12:42:42.751 INFO:teuthology.orchestra.console:Power off for smithi072 completed
2023-01-24T12:42:42.773 INFO:teuthology.orchestra.console:Power off for smithi085 completed
2023-01-24T12:42:42.897 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com
2023-01-24T12:42:42.915 INFO:teuthology.lock.ops:unlocked: smithi085.front.sepia.ceph.com
2023-01-24T12:42:43.096 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/dispatcher/__init__.py", line 63, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/dispatcher/supervisor.py", line 52, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/fog.py", line 271, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds