2023-03-27T06:37:58.820 INFO:root:teuthology version: 0.0.1.dev1+ge372916
2023-03-27T06:37:58.988 INFO:teuthology.lock.ops:Start node 'smithi003.front.sepia.ceph.com' reimaging
2023-03-27T06:37:58.990 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: reset os type and version on server
2023-03-27T06:37:58.990 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2023-03-27T06:37:59.010 INFO:teuthology.lock.ops:Node 'smithi003.front.sepia.ceph.com' reimaging is complete
2023-03-27T06:37:59.010 INFO:teuthology.lock.ops:Start node 'smithi145.front.sepia.ceph.com' reimaging
2023-03-27T06:37:59.011 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: reset os type and version on server
2023-03-27T06:37:59.011 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server
2023-03-27T06:37:59.035 INFO:teuthology.lock.ops:Node 'smithi145.front.sepia.ceph.com' reimaging is complete
2023-03-27T06:37:59.035 INFO:teuthology.lock.ops:Start node 'smithi190.front.sepia.ceph.com' reimaging
2023-03-27T06:37:59.035 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: reset os type and version on server
2023-03-27T06:37:59.036 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server
2023-03-27T06:37:59.062 INFO:teuthology.lock.ops:Node 'smithi190.front.sepia.ceph.com' reimaging is complete
2023-03-27T06:37:59.167 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of ubuntu 22.04
2023-03-27T06:37:59.194 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of ubuntu 22.04
2023-03-27T06:37:59.213 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of ubuntu 22.04
2023-03-27T06:37:59.515 INFO:teuthology.orchestra.console:Power off smithi145
2023-03-27T06:37:59.533 INFO:teuthology.orchestra.console:Power off smithi003
2023-03-27T06:37:59.546 INFO:teuthology.orchestra.console:Power off smithi190
2023-03-27T06:37:59.683 INFO:teuthology.orchestra.console:Power off for smithi003 completed
2023-03-27T06:37:59.688 INFO:teuthology.orchestra.console:Power off for smithi145 completed
2023-03-27T06:37:59.694 INFO:teuthology.orchestra.console:Power off for smithi190 completed
2023-03-27T06:37:59.785 INFO:teuthology.orchestra.console:Power on smithi003
2023-03-27T06:37:59.795 INFO:teuthology.orchestra.console:Power on smithi145
2023-03-27T06:37:59.804 INFO:teuthology.orchestra.console:Power on smithi190
2023-03-27T06:38:08.169 INFO:teuthology.orchestra.console:Power on for smithi003 completed
2023-03-27T06:38:08.174 INFO:teuthology.orchestra.console:Power on for smithi145 completed
2023-03-27T06:38:08.181 INFO:teuthology.orchestra.console:Power on for smithi190 completed
2023-03-27T06:38:08.270 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish
2023-03-27T06:38:08.274 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish
2023-03-27T06:38:08.282 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish
2023-03-27T06:49:54.885 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-03-27T06:50:03.973 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-03-27T06:50:13.029 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-03-27T06:50:22.085 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-03-27T06:50:31.145 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-03-27T06:51:10.277 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-03-27T06:51:19.333 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-03-27T06:51:28.389 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-03-27T06:51:37.149 WARNING:teuthology.provision.fog:timed out
2023-03-27T06:51:37.445 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-03-27T06:51:43.410 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-27T06:51:43.858 INFO:teuthology.provision.fog.smithi190:Node is ready
2023-03-27T06:51:43.906 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190.front.sepia.ceph.com
2023-03-27T06:51:43.954 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.15.190 smithi190.front.sepia.ceph.com smithi190
2023-03-27T06:51:44.290 INFO:teuthology.provision.fog.smithi190:Deploy complete!
2023-03-27T06:51:46.501 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-03-27T06:51:55.557 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.3
2023-03-27T06:53:01.573 WARNING:teuthology.provision.fog:timed out
2023-03-27T06:53:07.772 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-03-27T06:53:08.208 INFO:teuthology.provision.fog.smithi003:Node is ready
2023-03-27T06:53:08.262 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com
2023-03-27T06:53:08.296 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/dispatcher/supervisor.py", line 226, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/provision/fog.py", line 255, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/contextutil.py", line 135, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (60) after waiting for 900 seconds
2023-03-27T06:53:08.309 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003
2023-03-27T06:53:08.366 INFO:teuthology.nuke:Checking targets against current locks
2023-03-27T06:53:08.414 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-27T06:53:08.449 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-27T06:53:08.478 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-03-27T06:53:08.518 INFO:teuthology.orchestra.console:Power off smithi003
2023-03-27T06:53:08.580 INFO:teuthology.orchestra.console:Power off smithi145
2023-03-27T06:53:08.632 INFO:teuthology.orchestra.console:Power off smithi190
2023-03-27T06:53:08.682 INFO:teuthology.provision.fog.smithi003:Deploy complete!
2023-03-27T06:53:16.924 INFO:teuthology.orchestra.console:Power off for smithi003 completed
2023-03-27T06:53:16.967 INFO:teuthology.orchestra.console:Power off for smithi145 completed
2023-03-27T06:53:17.011 INFO:teuthology.orchestra.console:Power off for smithi190 completed
2023-03-27T06:53:17.089 INFO:teuthology.lock.ops:unlocked: smithi003.front.sepia.ceph.com
2023-03-27T06:53:17.133 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com
2023-03-27T06:53:17.164 INFO:teuthology.lock.ops:unlocked: smithi190.front.sepia.ceph.com
2023-03-27T06:53:17.549 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/dispatcher/supervisor.py", line 56, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/dispatcher/supervisor.py", line 226, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/provision/fog.py", line 255, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_main/teuthology/contextutil.py", line 135, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (60) after waiting for 900 seconds