2023-09-21T13:14:30.305 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-21T13:14:30.307 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T13:14:30.472 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f29b1a83d00>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f29b1735c10>, signals=[15])
2023-09-21T13:14:30.473 INFO:teuthology.lock.ops:Start node 'smithi103.front.sepia.ceph.com' reimaging
2023-09-21T13:14:30.474 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T13:14:30.474 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server
2023-09-21T13:14:30.495 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging
2023-09-21T13:14:30.495 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T13:14:30.496 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2023-09-21T13:14:30.497 DEBUG:teuthology.provision.fog.smithi103:FOG request: GET /host {"name": "smithi103"}
2023-09-21T13:14:30.520 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /host {"name": "smithi104"}
2023-09-21T13:14:30.571 DEBUG:teuthology.provision.fog.smithi103:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-09-21T13:14:30.589 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-09-21T13:14:30.614 DEBUG:teuthology.provision.fog.smithi103:Requesting image smithi_centos_9.stream (ID 10)
2023-09-21T13:14:30.614 DEBUG:teuthology.provision.fog.smithi103:FOG request: PUT /host/208 {"imageID": 10}
2023-09-21T13:14:30.650 DEBUG:teuthology.provision.fog.smithi104:Requesting image smithi_centos_9.stream (ID 10)
2023-09-21T13:14:30.651 DEBUG:teuthology.provision.fog.smithi104:FOG request: PUT /host/209 {"imageID": 10}
2023-09-21T13:14:30.669 INFO:teuthology.provision.fog.smithi103:Scheduling deploy of centos 9.stream
2023-09-21T13:14:30.669 DEBUG:teuthology.provision.fog.smithi103:FOG request: GET /task/active None
2023-09-21T13:14:30.700 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of centos 9.stream
2023-09-21T13:14:30.700 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-09-21T13:14:30.733 DEBUG:teuthology.provision.fog.smithi103:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T13:14:30.754 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T13:14:30.773 DEBUG:teuthology.provision.fog.smithi103:FOG request: POST /host/208/task {"taskTypeID": 1}
2023-09-21T13:14:30.786 DEBUG:teuthology.provision.fog.smithi104:FOG request: POST /host/209/task {"taskTypeID": 1}
2023-09-21T13:14:30.822 DEBUG:teuthology.provision.fog.smithi103:FOG request: GET /task/active None
2023-09-21T13:14:30.836 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-09-21T13:14:30.914 INFO:teuthology.orchestra.console.smithi103:Power off
2023-09-21T13:14:30.915 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:14:30.930 INFO:teuthology.orchestra.console.smithi104:Power off
2023-09-21T13:14:30.931 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:14:50.986 DEBUG:teuthology.orchestra.console.smithi103:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:14:50.987 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:14:50.996 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:14:50.996 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:15:11.058 DEBUG:teuthology.orchestra.console.smithi103:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:15:11.059 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:15:11.068 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:15:11.069 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:15:31.129 DEBUG:teuthology.orchestra.console.smithi103:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:15:31.130 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:31.139 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:15:31.140 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:32.139 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:15:32.146 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:15:36.142 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:36.150 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:37.252 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:15:37.258 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:15:41.254 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:41.261 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:42.364 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:15:42.369 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:15:46.370 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:46.378 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:47.481 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:15:47.486 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:15:51.486 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:51.494 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:52.597 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:15:52.602 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:15:56.602 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:56.609 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:15:57.712 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:15:57.718 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:01.714 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:01.721 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:02.824 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:02.828 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:06.826 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:06.832 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:07.935 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:07.940 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:11.938 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:11.945 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:13.047 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:13.053 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:17.050 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:17.056 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:18.159 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:18.164 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:22.164 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:22.171 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:23.273 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:23.279 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:27.278 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:27.288 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:28.391 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:28.396 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:32.394 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:32.401 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:33.503 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:33.508 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:37.506 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:37.513 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:38.614 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:38.621 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:42.618 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:42.625 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:16:43.728 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:16:43.729 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:16:43.730 ERROR:teuthology.orchestra.console.smithi103:Failed to power off
2023-09-21T13:16:43.730 INFO:teuthology.orchestra.console.smithi103:Power on
2023-09-21T13:16:43.730 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T13:16:43.738 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:16:43.740 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:16:43.740 ERROR:teuthology.orchestra.console.smithi104:Failed to power off
2023-09-21T13:16:43.741 INFO:teuthology.orchestra.console.smithi104:Power on
2023-09-21T13:16:43.741 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T13:17:03.800 DEBUG:teuthology.orchestra.console.smithi103:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:17:03.801 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T13:17:03.810 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:17:03.810 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T13:17:23.874 DEBUG:teuthology.orchestra.console.smithi103:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:17:23.875 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T13:17:23.885 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:17:23.886 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T13:17:43.945 DEBUG:teuthology.orchestra.console.smithi103:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:17:43.946 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:17:43.957 DEBUG:teuthology.orchestra.console.smithi104:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:17:43.958 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:17:44.956 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:17:44.965 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:17:48.962 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:17:48.969 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:17:50.072 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:17:50.078 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:17:54.078 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:17:54.085 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:17:55.187 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:17:55.193 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:17:59.190 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:17:59.197 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:00.300 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:00.304 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:04.306 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:04.313 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:05.415 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:05.422 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:09.420 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:09.428 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:10.530 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:10.536 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:14.534 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:14.542 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:15.646 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:15.652 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:19.654 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:19.661 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:20.764 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:20.770 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:24.766 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:24.776 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:25.881 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:25.887 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:29.887 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:29.896 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:30.999 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:31.005 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:35.002 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:35.008 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:36.113 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:36.116 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:40.118 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:40.125 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:41.228 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:41.233 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:45.230 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:45.236 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:46.340 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:46.344 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:50.345 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:50.353 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:51.456 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:51.461 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:55.458 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:55.465 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:18:56.568 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:18:56.569 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:18:56.569 DEBUG:teuthology.provision.fog.smithi103:Canceling deploy task with ID 499711
2023-09-21T13:18:56.569 DEBUG:teuthology.provision.fog.smithi103:FOG request: DELETE /task/cancel {"id": 499711}
2023-09-21T13:18:56.574 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:18:56.574 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:18:56.575 DEBUG:teuthology.provision.fog.smithi104:Canceling deploy task with ID 499712
2023-09-21T13:18:56.575 DEBUG:teuthology.provision.fog.smithi104:FOG request: DELETE /task/cancel {"id": 499712}
2023-09-21T13:18:56.625 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi104
2023-09-21T13:18:56.626 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T13:18:57.522 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=479680f177af44058e924480029b36f2
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi104
2023-09-21T13:18:57.526 INFO:teuthology.nuke:Checking targets against current locks
2023-09-21T13:18:57.585 DEBUG:teuthology.nuke:shortname: smithi103
2023-09-21T13:18:57.585 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T13:18:57.611 DEBUG:teuthology.nuke:shortname: smithi104
2023-09-21T13:18:57.611 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T13:18:57.615 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-09-21_13:12:05-rgw-wip-cbodley-testing-distro-default-smithi/7400177', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 13:14:29.477043', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBdtk5kpPG6eYNxsbUUhP94wDXvlRmA02PaokXMYSATdfsN+ggFZAqSpAnE008SiaWHo10lbXGPIxwUzYXOv798='}
2023-09-21T13:18:57.637 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-09-21_13:12:05-rgw-wip-cbodley-testing-distro-default-smithi/7400177', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 13:14:29.475332', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-09-21T13:18:57.670 INFO:teuthology.orchestra.console.smithi103:Power off
2023-09-21T13:18:57.670 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:18:57.702 INFO:teuthology.orchestra.console.smithi104:Power off
2023-09-21T13:18:57.703 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:19:17.741 DEBUG:teuthology.orchestra.console.smithi103:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:19:17.742 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:19:17.772 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:19:17.773 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:19:37.816 DEBUG:teuthology.orchestra.console.smithi103:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:19:37.817 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:19:37.844 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:19:37.845 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T13:19:57.889 DEBUG:teuthology.orchestra.console.smithi103:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:19:57.891 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:19:57.917 DEBUG:teuthology.orchestra.console.smithi104:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T13:19:57.918 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:19:58.901 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:19:58.926 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:02.903 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:02.928 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:04.016 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:04.038 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:08.018 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:08.039 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:09.129 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:09.148 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:13.134 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:13.150 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:14.244 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:14.260 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:18.250 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:18.264 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:19.367 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:19.373 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:23.370 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:23.379 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:24.482 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:24.490 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:28.486 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:28.494 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:29.596 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:29.603 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:33.602 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:33.611 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:34.714 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:34.720 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:38.718 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:38.725 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:39.828 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:39.833 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:43.830 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:43.838 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:44.941 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:44.948 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:48.946 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:48.955 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:50.058 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:50.067 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:54.064 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:54.071 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:55.174 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:20:55.182 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:20:59.178 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:20:59.186 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:21:00.289 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:21:00.297 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:21:04.294 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:21:04.301 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:21:05.403 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:21:05.409 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:21:09.406 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:21:09.414 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T13:21:10.518 DEBUG:teuthology.orchestra.console.smithi103:check power output: 
2023-09-21T13:21:10.519 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:21:10.519 ERROR:teuthology.orchestra.console.smithi103:Failed to power off
2023-09-21T13:21:10.524 DEBUG:teuthology.orchestra.console.smithi104:check power output: 
2023-09-21T13:21:10.524 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T13:21:10.525 ERROR:teuthology.orchestra.console.smithi104:Failed to power off
2023-09-21T13:21:10.672 INFO:teuthology.lock.ops:unlocked: smithi104.front.sepia.ceph.com
2023-09-21T13:21:10.689 INFO:teuthology.lock.ops:unlocked: smithi103.front.sepia.ceph.com
2023-09-21T13:21:10.884 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi104