2023-09-21T11:20:45.381 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-21T11:20:45.383 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T11:20:45.543 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f603702ec70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f6036dd7ca0>, signals=[15])
2023-09-21T11:20:45.544 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging
2023-09-21T11:20:45.544 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T11:20:45.545 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2023-09-21T11:20:45.572 INFO:teuthology.lock.ops:Start node 'smithi120.front.sepia.ceph.com' reimaging
2023-09-21T11:20:45.573 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T11:20:45.574 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server
2023-09-21T11:20:45.576 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /host {"name": "smithi104"}
2023-09-21T11:20:45.597 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /host {"name": "smithi120"}
2023-09-21T11:20:45.665 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T11:20:45.674 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-09-21T11:20:45.706 DEBUG:teuthology.provision.fog.smithi104:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T11:20:45.707 DEBUG:teuthology.provision.fog.smithi104:FOG request: PUT /host/209 {"imageID": 5}
2023-09-21T11:20:45.714 DEBUG:teuthology.provision.fog.smithi120:Requesting image smithi_rhel_8.6 (ID 5)
2023-09-21T11:20:45.714 DEBUG:teuthology.provision.fog.smithi120:FOG request: PUT /host/225 {"imageID": 5}
2023-09-21T11:20:45.749 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of rhel 8.6
2023-09-21T11:20:45.750 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-09-21T11:20:45.760 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.6
2023-09-21T11:20:45.760 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-09-21T11:20:45.808 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T11:20:45.820 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T11:20:45.841 DEBUG:teuthology.provision.fog.smithi104:FOG request: POST /host/209/task {"taskTypeID": 1}
2023-09-21T11:20:45.855 DEBUG:teuthology.provision.fog.smithi120:FOG request: POST /host/225/task {"taskTypeID": 1}
2023-09-21T11:20:45.889 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-09-21T11:20:45.906 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None
2023-09-21T11:20:45.993 INFO:teuthology.orchestra.console.smithi104:Power off
2023-09-21T11:20:45.994 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:20:46.007 INFO:teuthology.orchestra.console.smithi120:Power off
2023-09-21T11:20:46.007 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:21:06.069 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:21:06.070 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:21:06.080 DEBUG:teuthology.orchestra.console.smithi120:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:21:06.081 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:21:26.141 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:21:26.142 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:21:26.152 DEBUG:teuthology.orchestra.console.smithi120:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:21:26.153 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:21:46.214 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:21:46.215 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:21:46.223 DEBUG:teuthology.orchestra.console.smithi120:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:21:46.224 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:21:47.225 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:21:47.232 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:21:51.230 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:21:51.236 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:21:52.339 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:21:52.344 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:21:56.342 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:21:56.349 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:21:57.453 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:21:57.457 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:01.458 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:01.466 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:02.568 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:02.573 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:06.570 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:06.578 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:07.681 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:07.686 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:11.686 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:11.692 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:12.794 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:12.800 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:16.798 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:16.805 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:17.907 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:17.914 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:21.910 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:21.917 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:23.020 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:23.025 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:27.022 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:27.029 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:28.132 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:28.138 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:32.138 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:32.146 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:33.249 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:33.255 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:37.254 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:37.262 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:38.364 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:38.370 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:42.370 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:42.377 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:43.480 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:43.486 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:47.482 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:47.488 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:48.591 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:48.597 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:52.596 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:52.603 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:53.706 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:53.711 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:57.711 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:57.719 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:22:58.825 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:22:58.826 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T11:22:58.827 ERROR:teuthology.orchestra.console.smithi104:Failed to power off
2023-09-21T11:22:58.827 INFO:teuthology.orchestra.console.smithi104:Power on
2023-09-21T11:22:58.827 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T11:22:58.836 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:22:58.837 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T11:22:58.837 ERROR:teuthology.orchestra.console.smithi120:Failed to power off
2023-09-21T11:22:58.837 INFO:teuthology.orchestra.console.smithi120:Power on
2023-09-21T11:22:58.838 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T11:23:18.896 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:23:18.896 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T11:23:18.905 DEBUG:teuthology.orchestra.console.smithi120:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:23:18.905 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T11:23:38.965 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:23:38.966 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T11:23:38.975 DEBUG:teuthology.orchestra.console.smithi120:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:23:38.976 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T11:23:59.037 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:23:59.038 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:23:59.047 DEBUG:teuthology.orchestra.console.smithi120:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:23:59.047 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:00.047 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:00.053 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:04.050 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:04.056 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:05.158 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:05.165 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:09.162 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:09.168 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:10.271 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:10.276 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:14.277 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:14.283 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:15.386 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:15.393 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:19.391 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:19.398 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:20.500 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:20.505 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:24.506 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:24.512 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:25.615 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:25.620 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:29.620 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:29.626 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:30.729 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:30.734 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:34.734 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:34.741 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:35.843 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:35.848 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:39.846 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:39.853 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:40.956 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:40.961 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:44.958 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:44.964 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:46.066 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:46.072 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:50.070 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:50.077 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:51.180 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:51.186 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:24:55.187 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:55.194 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:24:56.295 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:24:56.303 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:25:00.300 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:25:00.307 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:25:01.410 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:25:01.417 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:25:05.414 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:25:05.420 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:25:06.523 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:25:06.529 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:25:10.527 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:25:10.534 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:25:11.636 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:25:11.637 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T11:25:11.638 DEBUG:teuthology.provision.fog.smithi104:Canceling deploy task with ID 499281
2023-09-21T11:25:11.638 DEBUG:teuthology.provision.fog.smithi104:FOG request: DELETE /task/cancel {"id": 499281}
2023-09-21T11:25:11.642 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:25:11.643 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T11:25:11.643 DEBUG:teuthology.provision.fog.smithi120:Canceling deploy task with ID 499282
2023-09-21T11:25:11.643 DEBUG:teuthology.provision.fog.smithi120:FOG request: DELETE /task/cancel {"id": 499282}
2023-09-21T11:25:11.692 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi120
2023-09-21T11:25:11.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T11:25:11.849 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1fdca80d0dfa4f39b2a4f97b67463f33
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi120
2023-09-21T11:25:11.852 INFO:teuthology.nuke:Checking targets against current locks
2023-09-21T11:25:11.909 DEBUG:teuthology.nuke:shortname: smithi104
2023-09-21T11:25:11.910 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T11:25:11.933 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399766', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 11:20:44.546770', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-09-21T11:25:11.937 DEBUG:teuthology.nuke:shortname: smithi120
2023-09-21T11:25:11.937 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T11:25:11.956 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399766', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 11:20:44.545078', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGa3dUXLF4DLd0peqpAryyvaYZFfZ1pWQayf9oAfPx6aQVpBwdkRQhpxuKJ4iGumbO4JsCLTIx8DsuNEX5v1XQg='}
2023-09-21T11:25:11.982 INFO:teuthology.orchestra.console.smithi104:Power off
2023-09-21T11:25:11.983 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:25:12.020 INFO:teuthology.orchestra.console.smithi120:Power off
2023-09-21T11:25:12.021 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:25:32.052 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:25:32.052 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:25:32.087 DEBUG:teuthology.orchestra.console.smithi120:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:25:32.088 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:25:52.124 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:25:52.125 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:25:52.157 DEBUG:teuthology.orchestra.console.smithi120:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:25:52.158 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T11:26:12.192 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:26:12.193 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:12.227 DEBUG:teuthology.orchestra.console.smithi120:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T11:26:12.228 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:13.202 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:13.238 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:17.204 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:17.239 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:18.314 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:18.349 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:22.318 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:22.351 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:23.428 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:23.458 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:27.434 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:27.460 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:28.544 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:28.568 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:32.550 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:32.570 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:33.660 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:33.680 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:37.667 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:37.681 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:38.777 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:38.790 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:42.782 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:42.792 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:43.893 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:43.900 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:47.898 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:47.905 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:49.008 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:49.013 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:53.010 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:53.017 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:54.120 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:54.125 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:26:58.126 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:58.133 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:26:59.235 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:26:59.240 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:27:03.238 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:03.245 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:04.348 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:27:04.353 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:27:08.351 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:08.358 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:09.460 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:27:09.466 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:27:13.466 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:13.472 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:14.575 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:27:14.581 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:27:18.579 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:18.586 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:19.689 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:27:19.694 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:27:23.695 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:23.702 DEBUG:teuthology.orchestra.console.smithi120:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T11:27:24.805 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T11:27:24.806 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T11:27:24.806 ERROR:teuthology.orchestra.console.smithi104:Failed to power off
2023-09-21T11:27:24.811 DEBUG:teuthology.orchestra.console.smithi120:check power output: 
2023-09-21T11:27:24.811 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T11:27:24.812 ERROR:teuthology.orchestra.console.smithi120:Failed to power off
2023-09-21T11:27:24.967 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com
2023-09-21T11:27:24.975 INFO:teuthology.lock.ops:unlocked: smithi104.front.sepia.ceph.com
2023-09-21T11:27:25.219 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi120