2023-09-21T12:33:13.013 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-21T12:33:13.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T12:33:13.159 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f8c3a4dfc40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f8c3a244c10>, signals=[15])
2023-09-21T12:33:13.161 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging
2023-09-21T12:33:13.161 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T12:33:13.161 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server
2023-09-21T12:33:13.181 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging
2023-09-21T12:33:13.182 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T12:33:13.182 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server
2023-09-21T12:33:13.183 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /host {"name": "smithi118"}
2023-09-21T12:33:13.206 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /host {"name": "smithi121"}
2023-09-21T12:33:13.262 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T12:33:13.274 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T12:33:13.317 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T12:33:13.317 DEBUG:teuthology.provision.fog.smithi118:FOG request: PUT /host/223 {"imageID": 5}
2023-09-21T12:33:13.321 DEBUG:teuthology.provision.fog.smithi121:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T12:33:13.321 DEBUG:teuthology.provision.fog.smithi121:FOG request: PUT /host/226 {"imageID": 5}
2023-09-21T12:33:13.365 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of rhel 8.6
2023-09-21T12:33:13.365 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None
2023-09-21T12:33:13.373 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of rhel 8.6
2023-09-21T12:33:13.373 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None
2023-09-21T12:33:13.447 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T12:33:13.455 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T12:33:13.487 DEBUG:teuthology.provision.fog.smithi118:FOG request: POST /host/223/task {"taskTypeID": 1}
2023-09-21T12:33:13.493 DEBUG:teuthology.provision.fog.smithi121:FOG request: POST /host/226/task {"taskTypeID": 1}
2023-09-21T12:33:13.542 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None
2023-09-21T12:33:13.544 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None
2023-09-21T12:33:13.664 INFO:teuthology.orchestra.console.smithi121:Power off
2023-09-21T12:33:13.664 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:33:13.677 INFO:teuthology.orchestra.console.smithi118:Power off
2023-09-21T12:33:13.678 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:33:33.732 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:33:33.732 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:33:33.746 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:33:33.746 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:33:53.807 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:33:53.808 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:33:53.817 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:33:53.817 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:34:13.877 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:34:13.878 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:13.886 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:34:13.887 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:14.886 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:14.895 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:18.890 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:18.897 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:20.000 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:20.005 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:24.004 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:24.011 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:25.114 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:25.119 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:29.119 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:29.126 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:30.229 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:30.234 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:34.235 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:34.241 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:35.344 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:35.349 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:39.346 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:39.352 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:40.455 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:40.463 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:44.458 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:44.465 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:45.566 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:45.573 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:49.569 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:49.575 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:50.678 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:50.682 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:54.683 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:54.690 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:55.793 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:34:55.797 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:34:59.798 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:34:59.805 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:00.908 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:35:00.913 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:35:04.913 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:04.920 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:06.022 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:35:06.028 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:35:10.025 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:10.032 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:11.134 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:35:11.139 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:35:15.138 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:15.145 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:16.247 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:35:16.253 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:35:20.250 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:20.256 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:21.359 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:35:21.363 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:35:25.362 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:25.368 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:35:26.470 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:35:26.471 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:35:26.472 ERROR:teuthology.orchestra.console.smithi121:Failed to power off
2023-09-21T12:35:26.472 INFO:teuthology.orchestra.console.smithi121:Power on
2023-09-21T12:35:26.472 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:35:26.480 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:35:26.481 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:35:26.481 ERROR:teuthology.orchestra.console.smithi118:Failed to power off
2023-09-21T12:35:26.482 INFO:teuthology.orchestra.console.smithi118:Power on
2023-09-21T12:35:26.482 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:35:46.539 DEBUG:teuthology.orchestra.console.smithi121:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:35:46.540 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:35:46.547 DEBUG:teuthology.orchestra.console.smithi118:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:35:46.548 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:36:06.609 DEBUG:teuthology.orchestra.console.smithi121:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:36:06.610 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:36:06.618 DEBUG:teuthology.orchestra.console.smithi118:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:36:06.619 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T12:36:26.684 DEBUG:teuthology.orchestra.console.smithi118:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:36:26.685 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:26.692 DEBUG:teuthology.orchestra.console.smithi121:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:36:26.692 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:27.700 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:36:27.701 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:36:31.702 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:31.709 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:32.811 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:36:32.817 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:36:36.817 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:36.824 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:37.926 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:36:37.932 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:36:41.930 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:41.937 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:43.040 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:36:43.046 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:36:47.042 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:47.050 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:48.153 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:36:48.159 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:36:52.158 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:52.165 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:53.267 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:36:53.273 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:36:57.270 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:57.277 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:36:58.380 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:36:58.384 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:02.382 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:02.388 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:03.490 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:37:03.496 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:07.494 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:07.501 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:08.604 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:37:08.609 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:12.606 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:12.615 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:13.717 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:37:13.723 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:17.722 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:17.729 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:18.832 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:37:18.837 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:22.836 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:22.843 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:23.945 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:37:23.950 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:27.950 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:27.956 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:29.059 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:37:29.063 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:33.065 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:33.072 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:34.174 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:37:34.179 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:38.176 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:38.182 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:37:39.284 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:37:39.285 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:37:39.285 DEBUG:teuthology.provision.fog.smithi118:Canceling deploy task with ID 499559
2023-09-21T12:37:39.286 DEBUG:teuthology.provision.fog.smithi118:FOG request: DELETE /task/cancel {"id": 499559}
2023-09-21T12:37:39.290 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:37:39.290 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:37:39.291 DEBUG:teuthology.provision.fog.smithi121:Canceling deploy task with ID 499558
2023-09-21T12:37:39.291 DEBUG:teuthology.provision.fog.smithi121:FOG request: DELETE /task/cancel {"id": 499558}
2023-09-21T12:37:39.346 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi118
2023-09-21T12:37:39.347 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T12:37:39.471 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1265cc5d829643a98da864338611010d
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi118
2023-09-21T12:37:39.473 INFO:teuthology.nuke:Checking targets against current locks
2023-09-21T12:37:39.538 DEBUG:teuthology.nuke:shortname: smithi118
2023-09-21T12:37:39.539 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T12:37:39.560 DEBUG:teuthology.nuke:shortname: smithi121
2023-09-21T12:37:39.561 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T12:37:39.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:33:12.217053', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRE42EXVWAQ9kqDjOQ6hOHfXJ+0+i8Bi+Rl2xk80yQLZTfeiJa0sLUT0v55bz/u7u8H3agZr2OeOhE92veP/Jk='}
2023-09-21T12:37:39.589 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:33:12.215544', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE1jn1oLBF86q1pnE7RuFXENl5s/X1tespMys9PDEb7ck5yZ2AevLKGrQjEtmL5Zl6NRsF5t8khgq2lSYP9XV8s='}
2023-09-21T12:37:39.613 INFO:teuthology.orchestra.console.smithi118:Power off
2023-09-21T12:37:39.614 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:37:39.650 INFO:teuthology.orchestra.console.smithi121:Power off
2023-09-21T12:37:39.650 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:37:59.688 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:37:59.689 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:37:59.715 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:37:59.716 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:38:19.761 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:38:19.762 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:38:19.784 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:38:19.784 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T12:38:39.833 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:38:39.834 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:39.851 DEBUG:teuthology.orchestra.console.smithi121:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T12:38:39.852 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:40.843 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:38:40.860 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:38:44.846 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:44.860 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:45.956 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:38:45.969 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:38:49.961 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:49.970 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:51.073 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:38:51.078 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:38:55.079 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:55.087 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:38:56.189 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:38:56.195 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:00.194 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:00.201 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:01.303 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:01.309 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:05.306 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:05.313 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:06.415 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:06.421 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:10.418 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:10.425 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:11.528 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:11.534 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:15.530 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:15.537 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:16.640 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:16.645 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:20.642 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:20.649 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:21.752 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:21.759 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:25.758 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:25.765 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:26.867 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:26.873 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:30.871 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:30.878 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:31.981 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:31.986 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:35.986 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:35.993 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:37.096 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:37.101 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:41.098 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:41.106 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:42.208 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:42.214 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:46.214 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:46.221 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:47.323 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:47.329 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:51.325 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:51.332 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T12:39:52.435 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T12:39:52.435 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:39:52.436 ERROR:teuthology.orchestra.console.smithi118:Failed to power off
2023-09-21T12:39:52.441 DEBUG:teuthology.orchestra.console.smithi121:check power output: 
2023-09-21T12:39:52.442 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T12:39:52.442 ERROR:teuthology.orchestra.console.smithi121:Failed to power off
2023-09-21T12:39:52.595 INFO:teuthology.lock.ops:unlocked: smithi118.front.sepia.ceph.com
2023-09-21T12:39:52.602 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com
2023-09-21T12:39:52.791 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi118