2023-11-04T04:25:45.046 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-04T04:25:45.048 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-04T04:25:45.181 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7faabd661a90>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7faabd409b80>, signals=[15])
2023-11-04T04:25:45.182 INFO:teuthology.lock.ops:Start node 'smithi114.front.sepia.ceph.com' reimaging
2023-11-04T04:25:45.183 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: reset os type and version on server
2023-11-04T04:25:45.183 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server
2023-11-04T04:25:45.208 INFO:teuthology.lock.ops:Start node 'smithi119.front.sepia.ceph.com' reimaging
2023-11-04T04:25:45.209 INFO:teuthology.lock.ops:Updating [smithi119.front.sepia.ceph.com]: reset os type and version on server
2023-11-04T04:25:45.209 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server
2023-11-04T04:25:45.210 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /host {"name": "smithi114"}
2023-11-04T04:25:45.233 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /host {"name": "smithi119"}
2023-11-04T04:25:45.287 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /image {"name": "smithi_ubuntu_20.04"}
2023-11-04T04:25:45.306 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /image {"name": "smithi_ubuntu_20.04"}
2023-11-04T04:25:45.327 DEBUG:teuthology.provision.fog.smithi114:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-11-04T04:25:45.328 DEBUG:teuthology.provision.fog.smithi114:FOG request: PUT /host/219 {"imageID": 2}
2023-11-04T04:25:45.347 DEBUG:teuthology.provision.fog.smithi119:Requesting image smithi_ubuntu_20.04 (ID 2)
2023-11-04T04:25:45.348 DEBUG:teuthology.provision.fog.smithi119:FOG request: PUT /host/224 {"imageID": 2}
2023-11-04T04:25:45.385 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of ubuntu 20.04
2023-11-04T04:25:45.385 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-11-04T04:25:45.405 INFO:teuthology.provision.fog.smithi119:Scheduling deploy of ubuntu 20.04
2023-11-04T04:25:45.406 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None
2023-11-04T04:25:45.458 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /tasktype {"name": "deploy"}
2023-11-04T04:25:45.472 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /tasktype {"name": "deploy"}
2023-11-04T04:25:45.499 DEBUG:teuthology.provision.fog.smithi114:FOG request: POST /host/219/task {"taskTypeID": 1}
2023-11-04T04:25:45.503 DEBUG:teuthology.provision.fog.smithi119:FOG request: POST /host/224/task {"taskTypeID": 1}
2023-11-04T04:25:45.558 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None
2023-11-04T04:25:45.559 DEBUG:teuthology.provision.fog.smithi119:FOG request: GET /task/active None
2023-11-04T04:25:45.662 INFO:teuthology.orchestra.console.smithi119:Power off
2023-11-04T04:25:45.663 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:25:45.676 INFO:teuthology.orchestra.console.smithi114:Power off
2023-11-04T04:25:45.676 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:26:05.730 DEBUG:teuthology.orchestra.console.smithi119:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:26:05.731 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:26:05.744 DEBUG:teuthology.orchestra.console.smithi114:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:26:05.745 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:26:25.800 DEBUG:teuthology.orchestra.console.smithi119:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:26:25.801 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:26:25.812 DEBUG:teuthology.orchestra.console.smithi114:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:26:25.812 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:26:45.868 DEBUG:teuthology.orchestra.console.smithi119:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:26:45.869 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:26:45.879 DEBUG:teuthology.orchestra.console.smithi114:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:26:45.880 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:26:46.878 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:26:46.888 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:26:50.882 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:26:50.889 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:26:51.992 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:26:51.997 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:26:55.997 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:26:56.004 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:26:57.106 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:26:57.111 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:01.110 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:01.117 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:02.219 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:02.225 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:06.223 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:06.230 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:07.333 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:07.338 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:11.338 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:11.345 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:12.448 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:12.452 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:16.452 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:16.460 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:17.562 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:17.568 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:21.566 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:21.575 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:22.677 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:22.685 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:26.682 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:26.689 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:27.792 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:27.817 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:31.818 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:31.826 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:32.929 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:32.934 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:36.934 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:36.941 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:38.044 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:38.051 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:42.046 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:42.053 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:43.156 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:43.161 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:47.158 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:47.165 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:48.268 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:48.274 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:52.270 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:52.278 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:53.380 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:53.385 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:57.382 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:57.390 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:27:58.494 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:27:58.496 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:27:58.497 ERROR:teuthology.orchestra.console.smithi119:Failed to power off
2023-11-04T04:27:58.497 INFO:teuthology.orchestra.console.smithi119:Power on
2023-11-04T04:27:58.497 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:27:58.509 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:27:58.510 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:27:58.510 ERROR:teuthology.orchestra.console.smithi114:Failed to power off
2023-11-04T04:27:58.510 INFO:teuthology.orchestra.console.smithi114:Power on
2023-11-04T04:27:58.511 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:28:18.568 DEBUG:teuthology.orchestra.console.smithi119:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:28:18.569 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:28:18.580 DEBUG:teuthology.orchestra.console.smithi114:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:28:18.580 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:28:38.640 DEBUG:teuthology.orchestra.console.smithi119:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:28:38.641 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:28:38.652 DEBUG:teuthology.orchestra.console.smithi114:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:28:38.652 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:28:58.709 DEBUG:teuthology.orchestra.console.smithi119:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:28:58.710 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:28:58.722 DEBUG:teuthology.orchestra.console.smithi114:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:28:58.722 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:28:59.722 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:28:59.729 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:03.726 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:03.735 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:04.837 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:04.842 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:08.842 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:08.850 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:09.952 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:09.958 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:13.954 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:13.961 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:15.064 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:15.069 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:19.068 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:19.075 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:20.178 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:20.183 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:24.183 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:24.190 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:25.293 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:25.299 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:29.298 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:29.307 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:30.410 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:30.415 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:34.414 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:34.421 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:35.524 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:35.529 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:39.526 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:39.533 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:40.635 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:40.641 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:44.640 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:44.648 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:45.752 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:45.759 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:49.754 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:49.763 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:50.866 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:50.873 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:54.870 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:54.877 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:55.980 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:29:55.985 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:29:59.986 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:29:59.993 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:30:01.096 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:30:01.102 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:30:05.102 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:30:05.109 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:30:06.212 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:30:06.217 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:30:10.214 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:30:10.221 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:30:11.324 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:30:11.325 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:30:11.325 DEBUG:teuthology.provision.fog.smithi119:Canceling deploy task with ID 582475
2023-11-04T04:30:11.325 DEBUG:teuthology.provision.fog.smithi119:FOG request: DELETE /task/cancel {"id": 582475}
2023-11-04T04:30:11.329 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:30:11.330 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:30:11.330 DEBUG:teuthology.provision.fog.smithi114:Canceling deploy task with ID 582476
2023-11-04T04:30:11.330 DEBUG:teuthology.provision.fog.smithi114:FOG request: DELETE /task/cancel {"id": 582476}
2023-11-04T04:30:11.395 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi119
2023-11-04T04:30:11.396 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-04T04:30:11.533 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=59c233bc82b34803b61b3bd969b4c3be
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi119
2023-11-04T04:30:11.536 INFO:teuthology.nuke:Checking targets against current locks
2023-11-04T04:30:11.597 DEBUG:teuthology.nuke:shortname: smithi114
2023-11-04T04:30:11.597 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-04T04:30:11.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2023-11-03_20:37:04-rgw-wip-eric-testing-1-distro-default-smithi/7446018', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 04:25:44.236677', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-11-04T04:30:11.624 DEBUG:teuthology.nuke:shortname: smithi119
2023-11-04T04:30:11.625 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-04T04:30:11.644 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2023-11-03_20:37:04-rgw-wip-eric-testing-1-distro-default-smithi/7446018', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 04:25:44.238278', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIv1z74QtOhtPoVF29QvLoawG3xrUUzV8r4y7zjFLrL+eGTe2lm7r1do1iNgmk+6r0juV+UJsG+lkbh3vHPz6y4='}
2023-11-04T04:30:11.666 INFO:teuthology.orchestra.console.smithi114:Power off
2023-11-04T04:30:11.667 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:30:11.702 INFO:teuthology.orchestra.console.smithi119:Power off
2023-11-04T04:30:11.703 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:30:31.732 DEBUG:teuthology.orchestra.console.smithi114:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:30:31.733 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:30:31.776 DEBUG:teuthology.orchestra.console.smithi119:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:30:31.777 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:30:51.805 DEBUG:teuthology.orchestra.console.smithi114:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:30:51.806 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:30:51.848 DEBUG:teuthology.orchestra.console.smithi119:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:30:51.849 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:31:11.876 DEBUG:teuthology.orchestra.console.smithi114:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:31:11.877 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:11.920 DEBUG:teuthology.orchestra.console.smithi119:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:31:11.921 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:12.891 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:12.928 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:16.892 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:16.930 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:18.004 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:18.039 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:22.006 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:22.040 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:23.117 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:23.150 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:27.122 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:27.151 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:28.232 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:28.259 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:32.236 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:32.261 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:33.347 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:33.370 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:37.352 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:37.371 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:38.462 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:38.480 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:42.467 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:42.481 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:43.577 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:43.590 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:47.583 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:47.591 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:48.694 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:48.700 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:52.698 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:52.706 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:53.809 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:53.815 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:31:57.814 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:57.821 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:31:58.923 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:31:58.929 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:32:02.929 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:02.936 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:04.039 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:32:04.044 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:32:08.042 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:08.049 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:09.152 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:32:09.159 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:32:13.154 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:13.161 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:14.264 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:32:14.270 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:32:18.266 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:18.274 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:19.376 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:32:19.382 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:32:23.382 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:23.389 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:32:24.490 DEBUG:teuthology.orchestra.console.smithi114:check power output: 
2023-11-04T04:32:24.491 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:32:24.492 ERROR:teuthology.orchestra.console.smithi114:Failed to power off
2023-11-04T04:32:24.498 DEBUG:teuthology.orchestra.console.smithi119:check power output: 
2023-11-04T04:32:24.498 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:32:24.498 ERROR:teuthology.orchestra.console.smithi119:Failed to power off
2023-11-04T04:32:24.634 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com
2023-11-04T04:32:24.641 INFO:teuthology.lock.ops:unlocked: smithi119.front.sepia.ceph.com
2023-11-04T04:32:24.870 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi119