2021-10-09T13:34:42.621 INFO:root:teuthology version: 1.1.0-f08b273e
2021-10-09T13:34:42.831 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba004.front.sepia.ceph.com' reimaging
2021-10-09T13:34:42.832 INFO:teuthology.lock.ops:Updating [ubuntu@gibba004.front.sepia.ceph.com]: reset os type and version on server
2021-10-09T13:34:42.832 INFO:teuthology.lock.ops:Updating gibba004.front.sepia.ceph.com on lock server
2021-10-09T13:34:42.849 INFO:teuthology.lock.ops:Node 'ubuntu@gibba004.front.sepia.ceph.com' reimaging is complete
2021-10-09T13:34:42.850 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba018.front.sepia.ceph.com' reimaging
2021-10-09T13:34:42.850 INFO:teuthology.lock.ops:Updating [ubuntu@gibba018.front.sepia.ceph.com]: reset os type and version on server
2021-10-09T13:34:42.851 INFO:teuthology.lock.ops:Updating gibba018.front.sepia.ceph.com on lock server
2021-10-09T13:34:42.868 INFO:teuthology.lock.ops:Node 'ubuntu@gibba018.front.sepia.ceph.com' reimaging is complete
2021-10-09T13:34:42.868 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba026.front.sepia.ceph.com' reimaging
2021-10-09T13:34:42.868 INFO:teuthology.lock.ops:Updating [ubuntu@gibba026.front.sepia.ceph.com]: reset os type and version on server
2021-10-09T13:34:42.869 INFO:teuthology.lock.ops:Updating gibba026.front.sepia.ceph.com on lock server
2021-10-09T13:34:42.885 INFO:teuthology.lock.ops:Node 'ubuntu@gibba026.front.sepia.ceph.com' reimaging is complete
2021-10-09T13:34:42.885 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba029.front.sepia.ceph.com' reimaging
2021-10-09T13:34:42.885 INFO:teuthology.lock.ops:Updating [ubuntu@gibba029.front.sepia.ceph.com]: reset os type and version on server
2021-10-09T13:34:42.886 INFO:teuthology.lock.ops:Updating gibba029.front.sepia.ceph.com on lock server
2021-10-09T13:34:42.901 INFO:teuthology.lock.ops:Node 'ubuntu@gibba029.front.sepia.ceph.com' reimaging is complete
2021-10-09T13:34:42.902 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba031.front.sepia.ceph.com' reimaging
2021-10-09T13:34:42.902 INFO:teuthology.lock.ops:Updating [ubuntu@gibba031.front.sepia.ceph.com]: reset os type and version on server
2021-10-09T13:34:42.902 INFO:teuthology.lock.ops:Updating gibba031.front.sepia.ceph.com on lock server
2021-10-09T13:34:42.919 INFO:teuthology.lock.ops:Node 'ubuntu@gibba031.front.sepia.ceph.com' reimaging is complete
2021-10-09T13:34:42.919 INFO:teuthology.lock.ops:Start node 'ubuntu@gibba045.front.sepia.ceph.com' reimaging
2021-10-09T13:34:42.920 INFO:teuthology.lock.ops:Updating [ubuntu@gibba045.front.sepia.ceph.com]: reset os type and version on server
2021-10-09T13:34:42.920 INFO:teuthology.lock.ops:Updating gibba045.front.sepia.ceph.com on lock server
2021-10-09T13:34:42.938 INFO:teuthology.lock.ops:Node 'ubuntu@gibba045.front.sepia.ceph.com' reimaging is complete
2021-10-09T13:34:43.096 INFO:teuthology.provision.fog.gibba004:Scheduling deploy of rhel 7.9
2021-10-09T13:34:43.146 INFO:teuthology.provision.fog.gibba026:Scheduling deploy of rhel 7.9
2021-10-09T13:34:43.157 INFO:teuthology.provision.fog.gibba018:Scheduling deploy of rhel 7.9
2021-10-09T13:34:43.189 INFO:teuthology.provision.fog.gibba029:Scheduling deploy of rhel 7.9
2021-10-09T13:34:43.193 INFO:teuthology.provision.fog.gibba031:Scheduling deploy of rhel 7.9
2021-10-09T13:34:43.227 INFO:teuthology.provision.fog.gibba045:Scheduling deploy of rhel 7.9
2021-10-09T13:34:43.498 INFO:teuthology.orchestra.console:Power off gibba004
2021-10-09T13:34:43.536 INFO:teuthology.orchestra.console:Power off gibba026
2021-10-09T13:34:43.575 INFO:teuthology.orchestra.console:Power off gibba018
2021-10-09T13:34:43.610 INFO:teuthology.orchestra.console:Power off gibba029
2021-10-09T13:34:43.649 INFO:teuthology.orchestra.console:Power off for gibba004 completed
2021-10-09T13:34:43.665 INFO:teuthology.orchestra.console:Power off gibba031
2021-10-09T13:34:43.685 INFO:teuthology.orchestra.console:Power off for gibba026 completed
2021-10-09T13:34:43.721 INFO:teuthology.orchestra.console:Power off for gibba018 completed
2021-10-09T13:34:43.746 INFO:teuthology.orchestra.console:Power off gibba045
2021-10-09T13:34:43.754 INFO:teuthology.orchestra.console:Power on gibba004
2021-10-09T13:34:43.763 INFO:teuthology.orchestra.console:Power off for gibba029 completed
2021-10-09T13:34:43.789 INFO:teuthology.orchestra.console:Power on gibba026
2021-10-09T13:34:43.816 INFO:teuthology.orchestra.console:Power off for gibba031 completed
2021-10-09T13:34:43.822 INFO:teuthology.orchestra.console:Power on gibba018
2021-10-09T13:34:43.864 INFO:teuthology.orchestra.console:Power on gibba029
2021-10-09T13:34:43.897 INFO:teuthology.orchestra.console:Power off for gibba045 completed
2021-10-09T13:34:43.918 INFO:teuthology.orchestra.console:Power on gibba031
2021-10-09T13:34:43.998 INFO:teuthology.orchestra.console:Power on gibba045
2021-10-09T13:34:48.085 INFO:teuthology.orchestra.console:Power on for gibba018 completed
2021-10-09T13:34:48.187 INFO:teuthology.provision.fog.gibba018:Waiting for deploy to finish
2021-10-09T13:34:48.254 INFO:teuthology.orchestra.console:Power on for gibba045 completed
2021-10-09T13:34:48.356 INFO:teuthology.provision.fog.gibba045:Waiting for deploy to finish
2021-10-09T13:34:52.136 INFO:teuthology.orchestra.console:Power on for gibba004 completed
2021-10-09T13:34:52.152 INFO:teuthology.orchestra.console:Power on for gibba026 completed
2021-10-09T13:34:52.231 INFO:teuthology.orchestra.console:Power on for gibba029 completed
2021-10-09T13:34:52.237 INFO:teuthology.provision.fog.gibba004:Waiting for deploy to finish
2021-10-09T13:34:52.253 INFO:teuthology.provision.fog.gibba026:Waiting for deploy to finish
2021-10-09T13:34:52.280 INFO:teuthology.orchestra.console:Power on for gibba031 completed
2021-10-09T13:34:52.333 INFO:teuthology.provision.fog.gibba029:Waiting for deploy to finish
2021-10-09T13:34:52.382 INFO:teuthology.provision.fog.gibba031:Waiting for deploy to finish
2021-10-09T13:39:06.854 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-09T13:39:10.708 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-09T13:39:10.729 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-09T13:39:17.864 INFO:teuthology.provision.fog.gibba018:Node is ready
2021-10-09T13:39:18.214 INFO:teuthology.orchestra.run.gibba018.stdout:gibba018.front.sepia.ceph.com
2021-10-09T13:39:18.555 INFO:teuthology.orchestra.run.gibba018.stdout:172.21.2.118 gibba018.front.sepia.ceph.com gibba018
2021-10-09T13:39:20.307 INFO:teuthology.provision.fog.gibba018:Deploy complete!
2021-10-09T13:39:22.145 INFO:teuthology.provision.fog.gibba004:Node is ready
2021-10-09T13:39:22.496 INFO:teuthology.orchestra.run.gibba004.stdout:gibba018
2021-10-09T13:39:22.832 INFO:teuthology.orchestra.run.gibba004.stdout:172.21.2.118 gibba018.front.sepia.ceph.com gibba018
2021-10-09T13:39:24.625 INFO:teuthology.provision.fog.gibba004:Deploy complete!
2021-10-09T13:39:26.530 INFO:teuthology.provision.fog.gibba026:Node is ready
2021-10-09T13:39:26.879 INFO:teuthology.orchestra.run.gibba026.stdout:gibba026.front.sepia.ceph.com
2021-10-09T13:39:27.215 INFO:teuthology.orchestra.run.gibba026.stdout:172.21.2.126 gibba026.front.sepia.ceph.com gibba026
2021-10-09T13:39:29.008 INFO:teuthology.provision.fog.gibba026:Deploy complete!
2021-10-09T13:39:33.541 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-09T13:39:34.173 INFO:teuthology.provision.fog.gibba029:Node is ready
2021-10-09T13:39:34.511 INFO:teuthology.orchestra.run.gibba029.stdout:gibba029.front.sepia.ceph.com
2021-10-09T13:39:34.846 INFO:teuthology.orchestra.run.gibba029.stdout:172.21.2.129 gibba029.front.sepia.ceph.com gibba029
2021-10-09T13:39:36.596 INFO:teuthology.provision.fog.gibba029:Deploy complete!
2021-10-09T13:39:39.560 INFO:teuthology.orchestra.run:Running command with timeout 600
2021-10-09T13:39:40.182 INFO:teuthology.provision.fog.gibba031:Node is ready
2021-10-09T13:39:40.522 INFO:teuthology.orchestra.run.gibba031.stdout:gibba031.front.sepia.ceph.com
2021-10-09T13:39:40.857 INFO:teuthology.orchestra.run.gibba031.stdout:172.21.2.131 gibba031.front.sepia.ceph.com gibba031
2021-10-09T13:39:42.601 INFO:teuthology.provision.fog.gibba031:Deploy complete!
2021-10-09T13:49:37.861 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/dispatcher/supervisor.py", line 163, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/lock/ops.py", line 315, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/provision/fog.py", line 244, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (60) after waiting for 900 seconds
2021-10-09T13:49:37.914 INFO:teuthology.nuke:Checking targets against current locks
2021-10-09T13:49:38.047 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-09T13:49:38.053 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-09T13:49:38.054 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-09T13:49:38.056 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-09T13:49:38.057 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-09T13:49:38.063 INFO:teuthology.task.internal.check_lock:Checking locks...
2021-10-09T13:49:38.125 INFO:teuthology.orchestra.console:Power off gibba004
2021-10-09T13:49:38.140 INFO:teuthology.orchestra.console:Power off gibba018
2021-10-09T13:49:38.150 INFO:teuthology.orchestra.console:Power off gibba026
2021-10-09T13:49:38.162 INFO:teuthology.orchestra.console:Power off gibba029
2021-10-09T13:49:38.173 INFO:teuthology.orchestra.console:Power off gibba031
2021-10-09T13:49:38.204 INFO:teuthology.orchestra.console:Power off gibba045
2021-10-09T13:49:46.553 INFO:teuthology.orchestra.console:Power off for gibba004 completed
2021-10-09T13:49:46.560 INFO:teuthology.orchestra.console:Power off for gibba018 completed
2021-10-09T13:49:46.567 INFO:teuthology.orchestra.console:Power off for gibba026 completed
2021-10-09T13:49:46.573 INFO:teuthology.orchestra.console:Power off for gibba031 completed
2021-10-09T13:49:46.587 INFO:teuthology.orchestra.console:Power off for gibba029 completed
2021-10-09T13:49:46.711 INFO:teuthology.lock.ops:unlocked: gibba004.front.sepia.ceph.com
2021-10-09T13:49:46.716 INFO:teuthology.lock.ops:unlocked: gibba018.front.sepia.ceph.com
2021-10-09T13:49:46.723 INFO:teuthology.lock.ops:unlocked: gibba031.front.sepia.ceph.com
2021-10-09T13:49:46.737 INFO:teuthology.lock.ops:unlocked: gibba029.front.sepia.ceph.com
2021-10-09T13:49:47.797 INFO:teuthology.lock.ops:unlocked: gibba026.front.sepia.ceph.com
2021-10-09T13:49:50.696 INFO:teuthology.orchestra.console:Power off for gibba045 completed
2021-10-09T13:49:50.843 INFO:teuthology.lock.ops:unlocked: gibba045.front.sepia.ceph.com
2021-10-09T13:49:50.843 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/virtualenv/bin/teuthology-dispatcher", line 33, in <module>
    sys.exit(load_entry_point('teuthology', 'console_scripts', 'teuthology-dispatcher')())
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/scripts/dispatcher.py", line 33, in main
    teuthology.dispatcher.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/dispatcher/__init__.py", line 61, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/dispatcher/supervisor.py", line 50, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/dispatcher/supervisor.py", line 163, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/lock/ops.py", line 315, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/provision/fog.py", line 86, in create
    self.wait_for_deploy_task(task_id)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/provision/fog.py", line 244, in wait_for_deploy_task
    while proceed():
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_f08b273eeec75318a83a4e8d7fb51841e4d71be0/teuthology/contextutil.py", line 133, in __call__
    raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (60) after waiting for 900 seconds