2022-12-19T21:27:35.424 INFO:root:teuthology version: 0.0.1.dev18+g19d18db
2022-12-19T21:27:35.648 INFO:teuthology.lock.ops:Start node 'smithi025.front.sepia.ceph.com' reimaging
2022-12-19T21:27:35.648 INFO:teuthology.lock.ops:Updating [smithi025.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:27:35.649 INFO:teuthology.lock.ops:Updating smithi025.front.sepia.ceph.com on lock server
2022-12-19T21:27:35.669 INFO:teuthology.lock.ops:Node 'smithi025.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:27:35.669 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging
2022-12-19T21:27:35.669 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:27:35.669 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server
2022-12-19T21:27:35.691 INFO:teuthology.lock.ops:Node 'smithi027.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:27:35.691 INFO:teuthology.lock.ops:Start node 'smithi032.front.sepia.ceph.com' reimaging
2022-12-19T21:27:35.691 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:27:35.691 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server
2022-12-19T21:27:35.718 INFO:teuthology.lock.ops:Node 'smithi032.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:27:35.718 INFO:teuthology.lock.ops:Start node 'smithi052.front.sepia.ceph.com' reimaging
2022-12-19T21:27:35.719 INFO:teuthology.lock.ops:Updating [smithi052.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:27:35.719 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server
2022-12-19T21:27:35.742 INFO:teuthology.lock.ops:Node 'smithi052.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:27:35.742 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging
2022-12-19T21:27:35.742 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:27:35.742 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server
2022-12-19T21:27:35.776 INFO:teuthology.lock.ops:Node 'smithi072.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:27:35.840 INFO:teuthology.provision.fog.smithi025:Scheduling deploy of ubuntu 18.04
2022-12-19T21:27:35.862 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of ubuntu 18.04
2022-12-19T21:27:35.886 INFO:teuthology.provision.fog.smithi052:Scheduling deploy of ubuntu 18.04
2022-12-19T21:27:35.891 INFO:teuthology.provision.fog.smithi032:Scheduling deploy of ubuntu 18.04
2022-12-19T21:27:35.929 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of ubuntu 18.04
2022-12-19T21:27:36.233 INFO:teuthology.orchestra.console:Power off smithi025
2022-12-19T21:27:36.283 INFO:teuthology.orchestra.console:Power off smithi027
2022-12-19T21:27:36.297 INFO:teuthology.orchestra.console:Power off smithi052
2022-12-19T21:27:36.348 INFO:teuthology.orchestra.console:Power off smithi032
2022-12-19T21:27:36.361 INFO:teuthology.orchestra.console:Power off smithi072
2022-12-19T21:27:36.375 INFO:teuthology.orchestra.console:Power off for smithi025 completed
2022-12-19T21:27:36.426 INFO:teuthology.orchestra.console:Power off for smithi027 completed
2022-12-19T21:27:36.438 INFO:teuthology.orchestra.console:Power off for smithi052 completed
2022-12-19T21:27:36.476 INFO:teuthology.orchestra.console:Power on smithi025
2022-12-19T21:27:36.488 INFO:teuthology.orchestra.console:Power off for smithi032 completed
2022-12-19T21:27:36.503 INFO:teuthology.orchestra.console:Power off for smithi072 completed
2022-12-19T21:27:36.527 INFO:teuthology.orchestra.console:Power on smithi027
2022-12-19T21:27:36.539 INFO:teuthology.orchestra.console:Power on smithi052
2022-12-19T21:27:36.589 INFO:teuthology.orchestra.console:Power on smithi032
2022-12-19T21:27:36.604 INFO:teuthology.orchestra.console:Power on smithi072
2022-12-19T21:27:40.796 INFO:teuthology.orchestra.console:Power on for smithi052 completed
2022-12-19T21:27:40.897 INFO:teuthology.provision.fog.smithi052:Waiting for deploy to finish
2022-12-19T21:27:44.848 INFO:teuthology.orchestra.console:Power on for smithi025 completed
2022-12-19T21:27:44.890 INFO:teuthology.orchestra.console:Power on for smithi027 completed
2022-12-19T21:27:44.948 INFO:teuthology.orchestra.console:Power on for smithi032 completed
2022-12-19T21:27:44.949 INFO:teuthology.provision.fog.smithi025:Waiting for deploy to finish
2022-12-19T21:27:44.978 INFO:teuthology.orchestra.console:Power on for smithi072 completed
2022-12-19T21:27:44.991 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish
2022-12-19T21:27:45.050 INFO:teuthology.provision.fog.smithi032:Waiting for deploy to finish
2022-12-19T21:27:45.079 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish
2022-12-19T21:34:02.535 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2022-12-19T21:34:11.591 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2022-12-19T21:34:20.651 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2022-12-19T21:34:21.799 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:34:29.735 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2022-12-19T21:34:30.855 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:34:38.791 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.52
2022-12-19T21:34:39.911 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:34:48.967 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:34:52.135 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2022-12-19T21:34:58.023 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:35:01.191 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2022-12-19T21:35:07.083 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:35:10.251 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2022-12-19T21:35:16.167 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:35:19.307 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2022-12-19T21:35:25.223 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:35:28.359 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2022-12-19T21:35:34.279 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:35:37.415 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2022-12-19T21:35:43.335 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:35:44.800 WARNING:teuthology.provision.fog:timed out
2022-12-19T21:35:46.471 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2022-12-19T21:35:50.892 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-12-19T21:36:24.071 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.72
2022-12-19T21:36:30.313 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-12-19T21:36:49.340 WARNING:teuthology.provision.fog:timed out
2022-12-19T21:37:13.740 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:37:22.791 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.27
2022-12-19T21:38:28.795 WARNING:teuthology.provision.fog:timed out
2022-12-19T21:38:34.882 INFO:teuthology.orchestra.run:Running command with timeout 600
2022-12-19T21:42:38.919 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 255, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (60) after waiting for 900 seconds
2022-12-19T21:42:38.987 INFO:teuthology.nuke:Checking targets against current locks
2022-12-19T21:42:39.041 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:42:39.067 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:42:39.094 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:42:39.115 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:42:39.127 INFO:teuthology.orchestra.console:Power off smithi025
2022-12-19T21:42:39.142 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:42:39.180 INFO:teuthology.orchestra.console:Power off smithi027
2022-12-19T21:42:39.218 INFO:teuthology.orchestra.console:Power off smithi052
2022-12-19T21:42:39.232 INFO:teuthology.orchestra.console:Power off smithi032
2022-12-19T21:42:39.275 INFO:teuthology.orchestra.console:Power off smithi072
2022-12-19T21:42:47.560 INFO:teuthology.orchestra.console:Power off for smithi027 completed
2022-12-19T21:42:47.581 INFO:teuthology.orchestra.console:Power off for smithi052 completed
2022-12-19T21:42:47.600 INFO:teuthology.orchestra.console:Power off for smithi032 completed
2022-12-19T21:42:47.630 INFO:teuthology.orchestra.console:Power off for smithi072 completed
2022-12-19T21:42:47.704 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com
2022-12-19T21:42:47.728 INFO:teuthology.lock.ops:unlocked: smithi052.front.sepia.ceph.com
2022-12-19T21:42:47.743 INFO:teuthology.lock.ops:unlocked: smithi032.front.sepia.ceph.com
2022-12-19T21:42:47.774 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com
2022-12-19T21:42:51.630 INFO:teuthology.orchestra.console:Power off for smithi025 completed
2022-12-19T21:42:51.776 INFO:teuthology.lock.ops:unlocked: smithi025.front.sepia.ceph.com
2022-12-19T21:42:58.582 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/__init__.py", line 63, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 52, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 255, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (60) after waiting for 900 seconds