2023-09-21T12:54:04.472 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-21T12:54:04.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T12:54:04.657 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f8dc3970ca0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f8dc3224c10>, signals=[15])
2023-09-21T12:54:04.659 INFO:teuthology.lock.ops:Start node 'smithi097.front.sepia.ceph.com' reimaging
2023-09-21T12:54:04.659 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T12:54:04.660 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server
2023-09-21T12:54:04.685 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging
2023-09-21T12:54:04.686 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T12:54:04.686 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2023-09-21T12:54:04.688 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /host {"name": "smithi097"}
2023-09-21T12:54:04.710 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging
2023-09-21T12:54:04.711 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T12:54:04.711 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server
2023-09-21T12:54:04.712 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /host {"name": "smithi121"}
2023-09-21T12:54:04.735 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /host {"name": "smithi125"}
2023-09-21T12:54:04.789 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T12:54:04.793 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T12:54:04.804 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T12:54:04.834 DEBUG:teuthology.provision.fog.smithi097:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T12:54:04.834 DEBUG:teuthology.provision.fog.smithi097:FOG request: PUT /host/202 {"imageID": 5}
2023-09-21T12:54:04.840 DEBUG:teuthology.provision.fog.smithi121:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T12:54:04.840 DEBUG:teuthology.provision.fog.smithi121:FOG request: PUT /host/226 {"imageID": 5}
2023-09-21T12:54:04.852 DEBUG:teuthology.provision.fog.smithi125:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T12:54:04.852 DEBUG:teuthology.provision.fog.smithi125:FOG request: PUT /host/230 {"imageID": 5}
2023-09-21T12:54:04.888 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of rhel 8.6
2023-09-21T12:54:04.888 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None
2023-09-21T12:54:04.898 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of rhel 8.6
2023-09-21T12:54:04.899 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None
2023-09-21T12:54:04.902 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of rhel 8.6
2023-09-21T12:54:04.902 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None
2023-09-21T12:54:04.952 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T12:54:04.963 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T12:54:04.967 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T12:54:04.987 DEBUG:teuthology.provision.fog.smithi121:FOG request: POST /host/226/task {"taskTypeID": 1}
2023-09-21T12:54:04.998 DEBUG:teuthology.provision.fog.smithi097:FOG request: POST /host/202/task {"taskTypeID": 1}
2023-09-21T12:54:05.006 DEBUG:teuthology.provision.fog.smithi125:FOG request: POST /host/230/task {"taskTypeID": 1}
2023-09-21T12:54:05.040 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None
2023-09-21T12:54:05.051 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None
2023-09-21T12:54:05.078 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None
2023-09-21T12:54:05.159 INFO:teuthology.orchestra.console.smithi121:Power off
2023-09-21T12:54:05.160 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:54:05.171 INFO:teuthology.orchestra.console.smithi097:Power off
2023-09-21T12:54:05.172 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:54:05.206 INFO:teuthology.orchestra.console.smithi125:Power off
2023-09-21T12:54:05.207 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:54:25.229 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:54:25.230 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:54:25.241 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:54:25.242 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:54:25.276 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:54:25.277 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:54:45.309 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:54:45.310 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:54:45.319 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:54:45.319 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:54:45.344 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:54:45.344 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:55:05.378 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:55:05.379 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:05.388 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:55:05.389 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:05.413 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:55:05.413 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:06.389 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:06.396 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:06.421 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:10.394 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:10.402 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:10.423 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:11.505 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:11.510 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:11.531 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:15.510 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:15.517 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:15.533 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:16.620 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:16.624 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:16.640 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:20.626 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:20.633 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:20.641 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:21.737 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:21.742 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:21.749 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:25.742 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:25.749 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:25.756 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:26.852 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:26.857 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:26.864 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:30.858 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:30.866 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:30.872 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:31.969 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:31.973 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:31.980 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:35.974 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:35.980 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:35.987 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:37.083 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:37.088 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:37.095 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:41.086 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:41.093 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:41.100 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:42.196 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:42.202 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:42.208 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:46.198 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:46.205 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:46.212 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:47.308 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:47.314 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:47.321 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:51.314 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:51.321 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:51.328 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:52.424 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:52.429 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:52.436 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:55:56.429 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:56.437 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:56.444 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:55:57.539 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:55:57.545 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:55:57.552 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:56:01.543 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:01.552 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:01.559 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:02.656 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:56:02.661 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:56:02.668 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:56:06.658 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:06.665 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:06.672 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:07.768 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:56:07.773 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:56:07.779 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:56:11.770 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:11.777 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:11.783 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:12.880 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:56:12.884 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:56:12.890 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:56:16.882 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:16.889 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:16.899 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:56:17.992 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:56:17.993 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:56:17.994 ERROR:teuthology.orchestra.console.smithi121:Failed to power off
2023-09-21T12:56:17.994 INFO:teuthology.orchestra.console.smithi121:Power on
2023-09-21T12:56:17.994 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:56:18.002 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:56:18.002 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:56:18.002 ERROR:teuthology.orchestra.console.smithi097:Failed to power off
2023-09-21T12:56:18.003 INFO:teuthology.orchestra.console.smithi097:Power on
2023-09-21T12:56:18.003 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:56:18.010 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:56:18.011 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:56:18.011 ERROR:teuthology.orchestra.console.smithi125:Failed to power off
2023-09-21T12:56:18.011 INFO:teuthology.orchestra.console.smithi125:Power on
2023-09-21T12:56:18.012 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:56:38.065 DEBUG:teuthology.orchestra.console.smithi121:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:56:38.066 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:56:38.077 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:56:38.077 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:56:38.085 DEBUG:teuthology.orchestra.console.smithi125:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:56:38.086 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:56:58.138 DEBUG:teuthology.orchestra.console.smithi121:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:56:58.139 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:56:58.148 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:56:58.149 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:56:58.155 DEBUG:teuthology.orchestra.console.smithi125:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:56:58.156 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:57:18.210 DEBUG:teuthology.orchestra.console.smithi121:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:57:18.212 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:18.221 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:57:18.222 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:18.230 DEBUG:teuthology.orchestra.console.smithi125:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:57:18.231 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:19.222 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:57:19.229 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:57:19.237 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:57:23.226 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:23.232 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:23.239 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:24.334 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:57:24.340 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:57:24.346 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:57:28.338 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:28.345 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:28.351 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:29.448 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:57:29.452 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:57:29.459 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:57:33.454 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:33.460 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:33.467 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:34.563 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:57:34.569 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:57:34.575 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:57:38.568 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:38.575 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:38.582 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:39.678 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:57:39.684 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:57:39.690 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:57:43.682 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:43.688 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:43.695 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:44.791 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:57:44.796 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:57:44.803 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:57:48.796 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:48.803 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:48.809 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:49.906 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:57:49.911 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:57:49.917 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:57:53.911 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:53.920 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:53.926 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:55.022 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:57:55.027 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:57:55.034 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:57:59.026 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:59.033 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:57:59.041 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:00.136 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:58:00.142 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:58:00.148 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:58:04.138 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:04.147 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:04.154 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:05.250 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:58:05.256 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:58:05.264 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:58:09.254 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:09.262 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:09.268 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:10.365 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:58:10.370 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:58:10.376 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:58:14.368 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:14.377 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:14.386 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:15.480 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:58:15.488 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:58:15.494 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:58:19.482 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:19.490 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:19.497 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:20.593 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:58:20.598 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:58:20.607 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:58:24.598 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:24.605 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:24.612 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:25.709 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:58:25.714 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:58:25.719 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:58:29.711 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:29.718 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:29.724 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:58:30.821 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:58:30.822 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:58:30.822 DEBUG:teuthology.provision.fog.smithi121:Canceling deploy task with ID 499641
2023-09-21T12:58:30.823 DEBUG:teuthology.provision.fog.smithi121:FOG request: DELETE /task/cancel {"id": 499641}
2023-09-21T12:58:30.828 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:58:30.828 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:58:30.829 DEBUG:teuthology.provision.fog.smithi097:Canceling deploy task with ID 499642
2023-09-21T12:58:30.829 DEBUG:teuthology.provision.fog.smithi097:FOG request: DELETE /task/cancel {"id": 499642}
2023-09-21T12:58:30.832 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:58:30.832 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:58:30.832 DEBUG:teuthology.provision.fog.smithi125:Canceling deploy task with ID 499643
2023-09-21T12:58:30.833 DEBUG:teuthology.provision.fog.smithi125:FOG request: DELETE /task/cancel {"id": 499643}
2023-09-21T12:58:30.878 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi121
2023-09-21T12:58:30.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T12:58:31.045 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=14567682fe3d4045b0189ba83ca97fe5
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi121
2023-09-21T12:58:31.048 INFO:teuthology.nuke:Checking targets against current locks
2023-09-21T12:58:31.103 DEBUG:teuthology.nuke:shortname: smithi097
2023-09-21T12:58:31.103 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T12:58:31.139 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:54:03.590942', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5PsAo33mTwEeo1Eu0EdUZ9KfA//4SDOy28XEJkXGfqNMeL3b7Qq8CbTO4CRaDEUE+6minPuzhmX0SCWe1ivyc='}
2023-09-21T12:58:31.141 DEBUG:teuthology.nuke:shortname: smithi121
2023-09-21T12:58:31.141 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T12:58:31.167 DEBUG:teuthology.nuke:shortname: smithi125
2023-09-21T12:58:31.167 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T12:58:31.170 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:54:03.592342', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE1jn1oLBF86q1pnE7RuFXENl5s/X1tespMys9PDEb7ck5yZ2AevLKGrQjEtmL5Zl6NRsF5t8khgq2lSYP9XV8s='}
2023-09-21T12:58:31.192 INFO:teuthology.orchestra.console.smithi097:Power off
2023-09-21T12:58:31.193 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:58:31.202 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399918', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:54:03.589290', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKrKaocHzvsx1FEGng3U74HTVChGLdJF4uALWtSJ6xbpJspvUuq2DiepEzmRIrn+LgzLzLIgo9uiaz6dsWDgbHk='}
2023-09-21T12:58:31.229 INFO:teuthology.orchestra.console.smithi121:Power off
2023-09-21T12:58:31.230 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:58:31.266 INFO:teuthology.orchestra.console.smithi125:Power off
2023-09-21T12:58:31.267 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:58:51.265 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:58:51.266 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:58:51.304 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:58:51.304 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:58:51.337 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:58:51.338 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:59:11.337 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:59:11.338 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:59:11.371 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:59:11.371 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:59:11.407 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:59:11.408 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:59:31.405 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:59:31.407 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:31.440 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:59:31.441 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:31.475 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:59:31.476 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:32.416 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:59:32.449 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:59:32.483 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:59:36.418 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:36.450 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:36.484 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:37.528 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:59:37.560 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:59:37.593 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:59:41.534 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:41.561 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:41.594 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:42.647 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:59:42.671 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:59:42.702 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:59:46.653 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:46.673 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:46.704 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:47.764 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:59:47.782 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:59:47.814 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:59:51.766 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:51.783 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:51.816 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:52.877 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:59:52.892 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:59:52.924 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T12:59:56.882 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:56.893 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:56.925 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:59:57.992 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T12:59:58.002 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:59:58.035 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:01.995 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:02.003 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:02.036 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:03.106 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:03.114 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:03.146 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:07.111 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:07.119 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:07.147 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:08.222 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:08.227 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:08.256 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:12.226 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:12.234 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:12.257 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:13.337 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:13.344 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:13.368 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:17.342 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:17.350 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:17.369 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:18.454 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:18.459 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:18.478 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:22.458 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:22.465 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:22.480 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:23.568 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:23.573 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:23.589 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:27.570 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:27.577 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:27.590 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:28.681 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:28.686 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:28.699 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:32.686 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:32.693 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:32.700 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:33.796 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:33.801 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:33.809 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:37.798 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:37.807 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:37.816 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:38.910 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:38.918 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:38.924 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:42.914 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:42.922 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:42.930 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:00:44.025 DEBUG:teuthology.orchestra.console.smithi097:check power output: 
2023-09-21T13:00:44.026 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:00:44.026 ERROR:teuthology.orchestra.console.smithi097:Failed to power off
2023-09-21T13:00:44.032 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T13:00:44.032 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:00:44.033 ERROR:teuthology.orchestra.console.smithi121:Failed to power off
2023-09-21T13:00:44.038 DEBUG:teuthology.orchestra.console.smithi125:check power output: 
2023-09-21T13:00:44.038 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:00:44.039 ERROR:teuthology.orchestra.console.smithi125:Failed to power off
2023-09-21T13:00:44.180 INFO:teuthology.lock.ops:unlocked: smithi097.front.sepia.ceph.com
2023-09-21T13:00:44.189 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com
2023-09-21T13:00:44.198 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com
2023-09-21T13:00:44.531 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi121