2023-11-04T04:53:37.402 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-04T04:53:37.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-04T04:53:37.545 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f0f30cb5b80>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f0f30968c10>, signals=[15])
2023-11-04T04:53:37.546 INFO:teuthology.lock.ops:Start node 'smithi102.front.sepia.ceph.com' reimaging
2023-11-04T04:53:37.547 INFO:teuthology.lock.ops:Updating [smithi102.front.sepia.ceph.com]: reset os type and version on server
2023-11-04T04:53:37.547 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server
2023-11-04T04:53:37.568 INFO:teuthology.lock.ops:Start node 'smithi105.front.sepia.ceph.com' reimaging
2023-11-04T04:53:37.568 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: reset os type and version on server
2023-11-04T04:53:37.569 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server
2023-11-04T04:53:37.570 DEBUG:teuthology.provision.fog.smithi102:FOG request: GET /host {"name": "smithi102"}
2023-11-04T04:53:37.592 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /host {"name": "smithi105"}
2023-11-04T04:53:37.639 DEBUG:teuthology.provision.fog.smithi102:FOG request: GET /image {"name": "smithi_ubuntu_22.04"}
2023-11-04T04:53:37.660 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /image {"name": "smithi_ubuntu_22.04"}
2023-11-04T04:53:37.682 DEBUG:teuthology.provision.fog.smithi102:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-11-04T04:53:37.683 DEBUG:teuthology.provision.fog.smithi102:FOG request: PUT /host/207 {"imageID": 6}
2023-11-04T04:53:37.698 DEBUG:teuthology.provision.fog.smithi105:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-11-04T04:53:37.699 DEBUG:teuthology.provision.fog.smithi105:FOG request: PUT /host/210 {"imageID": 6}
2023-11-04T04:53:37.732 INFO:teuthology.provision.fog.smithi102:Scheduling deploy of ubuntu 22.04
2023-11-04T04:53:37.732 DEBUG:teuthology.provision.fog.smithi102:FOG request: GET /task/active None
2023-11-04T04:53:37.741 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of ubuntu 22.04
2023-11-04T04:53:37.742 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None
2023-11-04T04:53:37.806 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /tasktype {"name": "deploy"}
2023-11-04T04:53:37.810 DEBUG:teuthology.provision.fog.smithi102:FOG request: GET /tasktype {"name": "deploy"}
2023-11-04T04:53:37.843 DEBUG:teuthology.provision.fog.smithi105:FOG request: POST /host/210/task {"taskTypeID": 1}
2023-11-04T04:53:37.846 DEBUG:teuthology.provision.fog.smithi102:FOG request: POST /host/207/task {"taskTypeID": 1}
2023-11-04T04:53:37.907 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None
2023-11-04T04:53:37.911 DEBUG:teuthology.provision.fog.smithi102:FOG request: GET /task/active None
2023-11-04T04:53:38.017 INFO:teuthology.orchestra.console.smithi102:Power off
2023-11-04T04:53:38.018 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:53:38.031 INFO:teuthology.orchestra.console.smithi105:Power off
2023-11-04T04:53:38.032 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:53:58.086 DEBUG:teuthology.orchestra.console.smithi102:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:53:58.087 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:53:58.105 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:53:58.106 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:54:18.161 DEBUG:teuthology.orchestra.console.smithi102:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:54:18.162 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:54:18.176 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:54:18.176 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:54:38.233 DEBUG:teuthology.orchestra.console.smithi102:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:54:38.234 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:38.244 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:54:38.244 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:39.243 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:54:39.251 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:54:43.246 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:43.254 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:44.356 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:54:44.361 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:54:48.358 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:48.365 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:49.468 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:54:49.473 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:54:53.472 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:53.481 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:54.584 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:54:54.589 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:54:58.587 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:58.595 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:54:59.697 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:54:59.704 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:03.702 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:03.714 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:04.816 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:04.825 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:08.820 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:08.829 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:09.930 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:09.938 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:13.934 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:13.941 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:15.045 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:15.050 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:19.046 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:19.055 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:20.158 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:20.163 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:24.162 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:24.170 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:25.273 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:25.280 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:29.278 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:29.287 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:30.390 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:30.395 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:34.394 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:34.401 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:35.504 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:35.509 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:39.508 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:39.516 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:40.618 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:40.625 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:44.622 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:44.630 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:45.732 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:45.739 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:49.734 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:49.741 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:55:50.844 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:55:50.845 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:55:50.845 ERROR:teuthology.orchestra.console.smithi102:Failed to power off
2023-11-04T04:55:50.846 INFO:teuthology.orchestra.console.smithi102:Power on
2023-11-04T04:55:50.846 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:55:50.853 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:55:50.854 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:55:50.854 ERROR:teuthology.orchestra.console.smithi105:Failed to power off
2023-11-04T04:55:50.855 INFO:teuthology.orchestra.console.smithi105:Power on
2023-11-04T04:55:50.855 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:56:10.913 DEBUG:teuthology.orchestra.console.smithi102:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:56:10.914 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:56:10.925 DEBUG:teuthology.orchestra.console.smithi105:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:56:10.926 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:56:30.988 DEBUG:teuthology.orchestra.console.smithi102:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:56:30.989 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:56:30.997 DEBUG:teuthology.orchestra.console.smithi105:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:56:30.998 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T04:56:51.056 DEBUG:teuthology.orchestra.console.smithi102:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:56:51.057 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:56:51.066 DEBUG:teuthology.orchestra.console.smithi105:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:56:51.066 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:56:52.066 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:56:52.075 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:56:56.071 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:56:56.080 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:56:57.183 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:56:57.188 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:01.188 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:01.194 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:02.297 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:02.303 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:06.302 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:06.310 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:07.414 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:07.421 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:11.418 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:11.425 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:12.528 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:12.533 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:16.533 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:16.540 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:17.642 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:17.649 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:21.646 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:21.653 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:22.756 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:22.761 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:26.762 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:26.771 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:27.874 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:27.880 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:31.878 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:31.885 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:32.989 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:32.994 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:36.994 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:37.002 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:38.104 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:38.109 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:42.106 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:42.113 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:43.216 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:43.223 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:47.218 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:47.229 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:48.332 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:48.339 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:52.334 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:52.341 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:53.444 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:53.453 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:57:57.446 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:57.454 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:57:58.557 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:57:58.563 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:58:02.562 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:58:02.569 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:58:03.672 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:58:03.673 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:58:03.673 DEBUG:teuthology.provision.fog.smithi102:Canceling deploy task with ID 582622
2023-11-04T04:58:03.674 DEBUG:teuthology.provision.fog.smithi102:FOG request: DELETE /task/cancel {"id": 582622}
2023-11-04T04:58:03.678 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:58:03.679 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T04:58:03.679 DEBUG:teuthology.provision.fog.smithi105:Canceling deploy task with ID 582621
2023-11-04T04:58:03.679 DEBUG:teuthology.provision.fog.smithi105:FOG request: DELETE /task/cancel {"id": 582621}
2023-11-04T04:58:03.726 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi102
2023-11-04T04:58:03.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-04T04:58:03.877 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7e95fed471424ce8be238e3c6c34c644
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi102
2023-11-04T04:58:03.880 INFO:teuthology.nuke:Checking targets against current locks
2023-11-04T04:58:03.943 DEBUG:teuthology.nuke:shortname: smithi102
2023-11-04T04:58:03.944 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-04T04:58:03.969 DEBUG:teuthology.nuke:shortname: smithi105
2023-11-04T04:58:03.969 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-04T04:58:03.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi/7447094', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 04:53:36.562952', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDolI1f/8OEvHXCzr/jWWDOcgGaGul0yhXs02+GQeO5LPIKLiVBHdO6wa5C2uVkqIoT2yZideWvronzdoQEjyZA='}
2023-11-04T04:58:03.996 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-04_04:30:51-fs-rishabh-2023nov3-testing-default-smithi/7447094', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 04:53:36.560989', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxyD+oef41D0gN6VPyH0FItRG170HMrLk1FDaSSw4/s2mOc946eKiqe9Gwvkmg6aPcpIjDI0dggHxubMW2QYJY='}
2023-11-04T04:58:04.028 INFO:teuthology.orchestra.console.smithi102:Power off
2023-11-04T04:58:04.029 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:58:04.061 INFO:teuthology.orchestra.console.smithi105:Power off
2023-11-04T04:58:04.062 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:58:24.098 DEBUG:teuthology.orchestra.console.smithi102:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:58:24.099 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:58:24.136 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:58:24.137 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:58:44.168 DEBUG:teuthology.orchestra.console.smithi102:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:58:44.169 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:58:44.209 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:58:44.210 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T04:59:04.240 DEBUG:teuthology.orchestra.console.smithi102:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:59:04.241 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:04.280 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T04:59:04.281 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:05.250 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:05.289 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:09.253 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:09.290 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:10.364 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:10.399 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:14.366 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:14.400 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:15.477 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:15.510 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:19.482 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:19.511 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:20.594 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:20.620 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:24.598 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:24.622 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:25.709 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:25.734 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:29.714 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:29.735 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:30.824 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:30.843 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:34.830 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:34.846 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:35.944 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:35.956 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:39.950 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:39.958 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:41.061 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:41.068 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:45.066 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:45.075 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:46.178 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:46.183 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:50.182 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:50.191 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:51.294 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:51.300 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T04:59:55.298 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:55.306 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T04:59:56.409 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T04:59:56.417 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T05:00:00.414 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T05:00:00.424 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T05:00:01.526 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T05:00:01.534 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T05:00:05.530 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T05:00:05.538 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T05:00:06.641 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T05:00:06.648 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T05:00:10.646 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T05:00:10.653 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T05:00:11.756 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T05:00:11.764 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T05:00:15.761 DEBUG:teuthology.orchestra.console.smithi102:pexpect command: ipmitool -H smithi102.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T05:00:15.769 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T05:00:16.872 DEBUG:teuthology.orchestra.console.smithi102:check power output: 
2023-11-04T05:00:16.873 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T05:00:16.873 ERROR:teuthology.orchestra.console.smithi102:Failed to power off
2023-11-04T05:00:16.878 DEBUG:teuthology.orchestra.console.smithi105:check power output: 
2023-11-04T05:00:16.878 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T05:00:16.879 ERROR:teuthology.orchestra.console.smithi105:Failed to power off
2023-11-04T05:00:17.033 INFO:teuthology.lock.ops:unlocked: smithi102.front.sepia.ceph.com
2023-11-04T05:00:17.042 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com
2023-11-04T05:00:17.275 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi102