2022-12-19T21:17:00.087 INFO:root:teuthology version: 0.0.1.dev18+g19d18db
2022-12-19T21:17:00.334 INFO:teuthology.lock.ops:Start node 'smithi080.front.sepia.ceph.com' reimaging
2022-12-19T21:17:00.335 INFO:teuthology.lock.ops:Updating [smithi080.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:17:00.335 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server
2022-12-19T21:17:00.353 INFO:teuthology.lock.ops:Node 'smithi080.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:17:00.353 INFO:teuthology.lock.ops:Start node 'smithi085.front.sepia.ceph.com' reimaging
2022-12-19T21:17:00.353 INFO:teuthology.lock.ops:Updating [smithi085.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:17:00.353 INFO:teuthology.lock.ops:Updating smithi085.front.sepia.ceph.com on lock server
2022-12-19T21:17:02.264 INFO:teuthology.lock.ops:Node 'smithi085.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:17:02.265 INFO:teuthology.lock.ops:Start node 'smithi086.front.sepia.ceph.com' reimaging
2022-12-19T21:17:02.265 INFO:teuthology.lock.ops:Updating [smithi086.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:17:02.265 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server
2022-12-19T21:17:02.285 INFO:teuthology.lock.ops:Node 'smithi086.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:17:02.285 INFO:teuthology.lock.ops:Start node 'smithi168.front.sepia.ceph.com' reimaging
2022-12-19T21:17:02.285 INFO:teuthology.lock.ops:Updating [smithi168.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:17:02.285 INFO:teuthology.lock.ops:Updating smithi168.front.sepia.ceph.com on lock server
2022-12-19T21:17:02.305 INFO:teuthology.lock.ops:Node 'smithi168.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:17:02.305 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging
2022-12-19T21:17:02.305 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server
2022-12-19T21:17:02.305 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server
2022-12-19T21:17:02.323 INFO:teuthology.lock.ops:Node 'smithi191.front.sepia.ceph.com' reimaging is complete
2022-12-19T21:17:02.472 INFO:teuthology.provision.fog.smithi080:Scheduling deploy of ubuntu 18.04
2022-12-19T21:17:02.480 INFO:teuthology.provision.fog.smithi085:Scheduling deploy of ubuntu 18.04
2022-12-19T21:17:02.490 INFO:teuthology.provision.fog.smithi086:Scheduling deploy of ubuntu 18.04
2022-12-19T21:17:02.506 INFO:teuthology.provision.fog.smithi168:Scheduling deploy of ubuntu 18.04
2022-12-19T21:17:02.510 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of ubuntu 18.04
2022-12-19T21:17:02.916 INFO:teuthology.orchestra.console:Power off smithi168
2022-12-19T21:17:02.930 INFO:teuthology.orchestra.console:Power off smithi085
2022-12-19T21:17:02.943 INFO:teuthology.orchestra.console:Power off smithi191
2022-12-19T21:17:02.990 INFO:teuthology.orchestra.console:Power off smithi080
2022-12-19T21:17:03.048 INFO:teuthology.orchestra.console:Power off smithi086
2022-12-19T21:17:03.081 INFO:teuthology.orchestra.console:Power off for smithi085 completed
2022-12-19T21:17:03.084 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2022-12-19T21:17:03.128 INFO:teuthology.orchestra.console:Power off for smithi080 completed
2022-12-19T21:17:03.182 INFO:teuthology.orchestra.console:Power on smithi085
2022-12-19T21:17:03.189 INFO:teuthology.orchestra.console:Power on smithi191
2022-12-19T21:17:03.196 INFO:teuthology.orchestra.console:Power off for smithi086 completed
2022-12-19T21:17:03.229 INFO:teuthology.orchestra.console:Power on smithi080
2022-12-19T21:17:03.296 INFO:teuthology.orchestra.console:Power on smithi086
2022-12-19T21:17:17.664 INFO:teuthology.orchestra.console:Power on for smithi085 completed
2022-12-19T21:17:17.723 INFO:teuthology.orchestra.console:Power on for smithi080 completed
2022-12-19T21:17:17.765 INFO:teuthology.provision.fog.smithi085:Waiting for deploy to finish
2022-12-19T21:17:17.769 INFO:teuthology.orchestra.console:Power on for smithi086 completed
2022-12-19T21:17:17.823 INFO:teuthology.provision.fog.smithi080:Waiting for deploy to finish
2022-12-19T21:17:17.870 INFO:teuthology.provision.fog.smithi086:Waiting for deploy to finish
2022-12-19T21:17:27.888 INFO:teuthology.orchestra.console:Power on for smithi191 completed
2022-12-19T21:17:27.989 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish
2022-12-19T21:19:39.915 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (15) after waiting for 60.0 seconds
2022-12-19T21:19:39.915 ERROR:teuthology.orchestra.console:Failed to power off smithi168
2022-12-19T21:19:39.916 INFO:teuthology.orchestra.console:Power on smithi168
2022-12-19T21:22:34.024 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (15) after waiting for 60.0 seconds
2022-12-19T21:22:34.098 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/orchestra/console.py", line 243, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi168
2022-12-19T21:22:34.168 INFO:teuthology.nuke:Checking targets against current locks
2022-12-19T21:22:34.223 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:22:34.247 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:22:34.269 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:22:34.294 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:22:34.304 INFO:teuthology.orchestra.console:Power off smithi080
2022-12-19T21:22:34.319 INFO:teuthology.task.internal.check_lock:Checking locks...
2022-12-19T21:22:34.346 INFO:teuthology.orchestra.console:Power off smithi085
2022-12-19T21:22:34.389 INFO:teuthology.orchestra.console:Power off smithi086
2022-12-19T21:22:34.425 INFO:teuthology.orchestra.console:Power off smithi191
2022-12-19T21:22:34.437 INFO:teuthology.orchestra.console:Power off smithi168
2022-12-19T21:22:43.689 INFO:teuthology.orchestra.console:Power off for smithi080 completed
2022-12-19T21:22:43.772 INFO:teuthology.orchestra.console:Power off for smithi086 completed
2022-12-19T21:22:58.839 INFO:teuthology.orchestra.console:Power off for smithi085 completed
2022-12-19T21:23:20.131 INFO:teuthology.lock.ops:unlocked: smithi080.front.sepia.ceph.com
2022-12-19T21:23:20.133 INFO:teuthology.lock.ops:unlocked: smithi086.front.sepia.ceph.com
2022-12-19T21:23:20.162 INFO:teuthology.lock.ops:unlocked: smithi085.front.sepia.ceph.com
2022-12-19T21:23:20.206 INFO:teuthology.orchestra.console:Power off for smithi191 completed
2022-12-19T21:23:20.350 INFO:teuthology.lock.ops:unlocked: smithi191.front.sepia.ceph.com
2022-12-19T21:23:56.967 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.80
2022-12-19T21:23:56.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2022-12-19T21:23:56.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.86
2022-12-19T21:23:56.968 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2022-12-19T21:24:06.055 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.80
2022-12-19T21:24:06.056 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2022-12-19T21:24:06.056 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.86
2022-12-19T21:24:06.056 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2022-12-19T21:24:18.471 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.86
2022-12-19T21:24:18.472 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2022-12-19T21:24:18.472 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.80
2022-12-19T21:24:18.472 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2022-12-19T21:24:27.687 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.86
2022-12-19T21:24:27.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2022-12-19T21:24:27.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.80
2022-12-19T21:24:27.688 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2022-12-19T21:24:36.903 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.86
2022-12-19T21:24:36.904 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2022-12-19T21:24:36.904 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.80
2022-12-19T21:24:36.904 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2022-12-19T21:24:49.159 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2022-12-19T21:24:49.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.80
2022-12-19T21:24:49.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2022-12-19T21:24:49.160 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.86
2022-12-19T21:24:58.407 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.191
2022-12-19T21:24:58.408 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.80
2022-12-19T21:24:58.408 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.85
2022-12-19T21:24:58.408 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.86
2022-12-19T21:25:04.420 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.86
2022-12-19T21:25:04.421 WARNING:teuthology.provision.fog:[Errno None] Unable to connect to port 22 on 172.21.15.80
2022-12-19T21:25:04.491 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:04.526 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:05.606 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:05.642 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:06.717 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:06.722 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:07.652 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (15) after waiting for 60.0 seconds
2022-12-19T21:25:07.652 ERROR:teuthology.orchestra.console:Failed to power off smithi168
2022-12-19T21:25:18.833 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:18.834 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:18.844 INFO:teuthology.lock.ops:unlocked: smithi168.front.sepia.ceph.com
2022-12-19T21:25:26.079 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:26.083 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:26.170 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:26.177 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:27.163 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:27.205 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:27.243 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:27.251 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:28.327 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:28.330 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:28.333 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:28.418 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:41.530 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:41.534 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:41.537 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:41.541 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:42.612 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:42.621 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:42.645 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:42.671 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:43.731 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:43.731 WARNING:teuthology.provision.fog:'connect to smithi085.front.sepia.ceph.com' reached maximum tries (10) after waiting for 10 seconds
2022-12-19T21:25:43.742 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:43.759 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:43.790 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:49.883 ERROR:teuthology.orchestra.connection:Error authenticating with smithi085.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:49.887 ERROR:teuthology.orchestra.connection:Error authenticating with smithi191.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:49.887 WARNING:teuthology.provision.fog:'connect to smithi191.front.sepia.ceph.com' reached maximum tries (10) after waiting for 10 seconds
2022-12-19T21:25:49.889 ERROR:teuthology.orchestra.connection:Error authenticating with smithi080.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:49.893 ERROR:teuthology.orchestra.connection:Error authenticating with smithi086.front.sepia.ceph.com: Authentication failed.
2022-12-19T21:25:50.575 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/__init__.py", line 63, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 52, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/dispatcher/supervisor.py", line 209, in reimage
    reimaged = reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/lock/ops.py", line 324, in reimage_machines
    log.info("Node '%s' reimaging is complete", machine)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_19d18db866afdc31fd6586f558fc29b95a87ccfb/teuthology/orchestra/console.py", line 243, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi168