2023-01-24T13:04:57.124 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1
2023-01-24T13:04:57.244 INFO:teuthology.lock.ops:Start node 'smithi083.front.sepia.ceph.com' reimaging
2023-01-24T13:04:57.245 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: reset os type and version on server
2023-01-24T13:04:57.246 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server
2023-01-24T13:04:57.263 INFO:teuthology.lock.ops:Node 'smithi083.front.sepia.ceph.com' reimaging is complete
2023-01-24T13:04:57.264 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging
2023-01-24T13:04:57.264 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server
2023-01-24T13:04:57.264 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server
2023-01-24T13:04:57.283 INFO:teuthology.lock.ops:Node 'smithi140.front.sepia.ceph.com' reimaging is complete
2023-01-24T13:04:57.431 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of rhel 8.6
2023-01-24T13:04:57.434 INFO:teuthology.provision.fog.smithi083:Scheduling deploy of rhel 8.6
2023-01-24T13:04:57.667 INFO:teuthology.orchestra.console:Power off smithi140
2023-01-24T13:04:57.679 INFO:teuthology.orchestra.console:Power off smithi083
2023-01-24T13:04:57.807 INFO:teuthology.orchestra.console:Power off for smithi140 completed
2023-01-24T13:04:57.819 INFO:teuthology.orchestra.console:Power off for smithi083 completed
2023-01-24T13:04:57.909 INFO:teuthology.orchestra.console:Power on smithi140
2023-01-24T13:04:57.921 INFO:teuthology.orchestra.console:Power on smithi083
2023-01-24T13:05:06.288 INFO:teuthology.orchestra.console:Power on for smithi140 completed
2023-01-24T13:05:06.293 INFO:teuthology.orchestra.console:Power on for smithi083 completed
2023-01-24T13:05:06.390 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish
2023-01-24T13:05:06.394 INFO:teuthology.provision.fog.smithi083:Waiting for deploy to finish
2023-01-24T13:07:37.526 ERROR:teuthology.orchestra.connection:Error authenticating with smithi083.front.sepia.ceph.com: Authentication failed.
2023-01-24T13:08:10.732 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:08:19.784 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:08:28.840 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:08:37.896 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:08:38.530 WARNING:teuthology.provision.fog:timed out
2023-01-24T13:08:46.952 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:09:02.952 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.83
2023-01-24T13:09:12.008 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.83
2023-01-24T13:09:18.609 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-01-24T13:09:18.914 INFO:teuthology.provision.fog.smithi083:Node is ready
2023-01-24T13:09:18.931 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083.front.sepia.ceph.com
2023-01-24T13:09:18.990 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83 smithi083.front.sepia.ceph.com smithi083
2023-01-24T13:09:19.374 INFO:teuthology.provision.fog.smithi083:Deploy complete!
2023-01-24T13:09:52.958 WARNING:teuthology.provision.fog:timed out
2023-01-24T13:10:17.448 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:10:26.504 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:10:35.564 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:10:44.652 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:10:53.708 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:11:02.760 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:11:11.816 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:11:20.872 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:11:29.928 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:11:38.988 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:11:48.040 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:11:57.096 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:12:06.152 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:12:15.208 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:12:24.268 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:12:33.324 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:12:42.380 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:12:51.436 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:13:00.492 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:13:09.548 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:13:18.604 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:13:27.656 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:13:36.712 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:13:45.772 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:13:54.828 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:14:03.884 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:14:12.936 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:14:21.992 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:14:31.052 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:14:40.104 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:14:49.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:14:58.216 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:15:07.272 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:15:16.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:15:25.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:15:34.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:15:43.496 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:15:52.556 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:16:01.612 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:16:10.668 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:16:19.724 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:16:28.776 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:16:37.836 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:16:46.888 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:16:55.944 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:17:05.000 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:17:14.056 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:17:23.112 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:17:32.172 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:17:41.228 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:17:50.284 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:17:59.336 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:18:08.392 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:18:17.448 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:18:26.504 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:18:35.560 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:18:44.616 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:18:53.672 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:19:02.732 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:19:11.788 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:19:20.840 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:19:29.896 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:19:38.956 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:19:48.008 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:19:57.068 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:20:06.120 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:20:15.176 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:20:24.232 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:20:33.292 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:20:42.348 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:20:51.400 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:21:00.456 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:21:09.516 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:21:18.600 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:21:27.656 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:21:36.716 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:21:45.772 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:21:54.824 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:22:03.881 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:22:12.940 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:22:21.996 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:22:31.052 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:22:40.108 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:22:49.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:22:58.216 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:23:07.276 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:23:16.328 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:23:25.384 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:23:34.440 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:23:43.500 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:23:52.552 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:24:01.612 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:24:10.664 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:24:19.720 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.140
2023-01-24T13:24:19.722 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/fog.py", line 271, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds
2023-01-24T13:24:19.789 INFO:teuthology.nuke:Checking targets against current locks
2023-01-24T13:24:19.838 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-24T13:24:19.866 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-01-24T13:24:19.918 INFO:teuthology.orchestra.console:Power off smithi083
2023-01-24T13:24:19.950 INFO:teuthology.orchestra.console:Power off smithi140
2023-01-24T13:24:32.386 INFO:teuthology.orchestra.console:Power off for smithi083 completed
2023-01-24T13:24:32.417 INFO:teuthology.orchestra.console:Power off for smithi140 completed
2023-01-24T13:24:32.545 INFO:teuthology.lock.ops:unlocked: smithi083.front.sepia.ceph.com
2023-01-24T13:24:32.568 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com
2023-01-24T13:24:32.769 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/dispatcher/__init__.py", line 63, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/dispatcher/supervisor.py", line 52, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/provision/fog.py", line 271, in _wait_for_ready
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds