2023-09-21T12:26:26.668 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-21T12:26:26.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-21T12:26:26.810 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffa9bbc7b80>, signals=[15]) 2023-09-21T12:26:26.812 INFO:teuthology.lock.ops:Start node 'smithi097.front.sepia.ceph.com' reimaging 2023-09-21T12:26:26.812 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: reset os type and version on server 2023-09-21T12:26:26.813 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-09-21T12:26:26.839 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging 2023-09-21T12:26:26.839 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server 2023-09-21T12:26:26.840 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-09-21T12:26:26.841 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /host {"name": "smithi097"} 2023-09-21T12:26:26.864 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /host {"name": "smithi125"} 2023-09-21T12:26:26.916 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-21T12:26:26.943 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-09-21T12:26:26.965 DEBUG:teuthology.provision.fog.smithi097:Requesting image smithi_centos_8.stream (ID 3) 2023-09-21T12:26:26.965 DEBUG:teuthology.provision.fog.smithi097:FOG request: PUT /host/202 {"imageID": 3} 2023-09-21T12:26:26.983 DEBUG:teuthology.provision.fog.smithi125:Requesting image smithi_centos_8.stream (ID 3) 2023-09-21T12:26:26.983 DEBUG:teuthology.provision.fog.smithi125:FOG request: PUT /host/230 {"imageID": 3} 2023-09-21T12:26:27.028 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of centos 8.stream 2023-09-21T12:26:27.028 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-09-21T12:26:27.034 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of centos 8.stream 2023-09-21T12:26:27.034 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-09-21T12:26:27.095 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /tasktype {"name": "deploy"} 2023-09-21T12:26:27.099 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /tasktype {"name": "deploy"} 2023-09-21T12:26:27.135 DEBUG:teuthology.provision.fog.smithi097:FOG request: POST /host/202/task {"taskTypeID": 1} 2023-09-21T12:26:27.146 DEBUG:teuthology.provision.fog.smithi125:FOG request: POST /host/230/task {"taskTypeID": 1} 2023-09-21T12:26:27.201 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-09-21T12:26:27.292 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-09-21T12:26:27.360 INFO:teuthology.orchestra.console.smithi097:Power off 2023-09-21T12:26:27.361 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:26:27.397 INFO:teuthology.orchestra.console.smithi125:Power off 2023-09-21T12:26:27.398 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:26:47.428 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:26:47.429 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:26:47.467 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:26:47.468 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:27:07.501 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:27:07.502 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:27:07.540 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:27:07.540 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:27:27.572 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:27:27.572 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:27.607 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:27:27.608 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:28.586 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:27:28.616 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:27:32.590 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:32.617 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:33.701 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:27:33.728 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:27:37.706 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:37.729 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:38.817 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:27:38.839 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:27:42.822 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:42.840 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:43.931 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:27:43.949 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:27:47.935 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:47.950 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:49.046 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:27:49.059 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:27:53.050 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:53.060 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:54.159 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:27:54.168 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:27:58.162 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:58.170 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:27:59.273 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:27:59.278 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:03.278 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:03.285 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:04.388 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:28:04.393 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:08.390 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:08.397 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:09.499 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:28:09.510 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:13.502 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:13.511 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:14.612 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:28:14.620 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:18.618 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:18.626 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:19.729 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:28:19.734 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:23.734 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:23.742 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:24.844 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:28:24.850 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:28.850 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:28.858 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:29.961 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:28:29.968 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:33.966 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:33.974 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:35.076 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:28:35.082 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:39.082 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:39.089 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:28:40.192 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:28:40.193 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T12:28:40.193 ERROR:teuthology.orchestra.console.smithi097:Failed to power off 2023-09-21T12:28:40.194 INFO:teuthology.orchestra.console.smithi097:Power on 2023-09-21T12:28:40.194 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T12:28:40.204 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:28:40.205 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T12:28:40.206 ERROR:teuthology.orchestra.console.smithi125:Failed to power off 2023-09-21T12:28:40.206 INFO:teuthology.orchestra.console.smithi125:Power on 2023-09-21T12:28:40.206 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T12:29:00.265 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:29:00.266 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T12:29:00.277 DEBUG:teuthology.orchestra.console.smithi125:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:29:00.278 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T12:29:20.338 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:29:20.339 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T12:29:20.348 DEBUG:teuthology.orchestra.console.smithi125:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:29:20.348 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T12:29:40.408 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:29:40.408 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:29:40.417 DEBUG:teuthology.orchestra.console.smithi125:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:29:40.418 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:29:41.417 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:29:41.424 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:29:45.422 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:29:45.429 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:29:46.532 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:29:46.537 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:29:50.538 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:29:50.546 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:29:51.650 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:29:51.656 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:29:55.654 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:29:55.661 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:29:56.763 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:29:56.768 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:00.766 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:00.772 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:01.875 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:01.879 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:05.880 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:05.887 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:06.990 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:06.995 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:10.995 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:11.002 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:12.105 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:12.110 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:16.106 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:16.113 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:17.214 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:17.220 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:21.218 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:21.225 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:22.328 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:22.334 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:26.330 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:26.661 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:27.764 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:27.774 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:31.766 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:31.775 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:32.875 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:32.884 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:36.880 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:36.887 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:37.990 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:37.995 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:41.994 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:42.001 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:43.104 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:43.111 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:47.106 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:47.113 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:48.216 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:48.222 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:52.218 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:52.226 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:30:53.328 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:30:53.329 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T12:30:53.329 DEBUG:teuthology.provision.fog.smithi097:Canceling deploy task with ID 499526 2023-09-21T12:30:53.329 DEBUG:teuthology.provision.fog.smithi097:FOG request: DELETE /task/cancel {"id": 499526} 2023-09-21T12:30:53.334 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:30:53.335 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T12:30:53.335 DEBUG:teuthology.provision.fog.smithi125:Canceling deploy task with ID 499527 2023-09-21T12:30:53.335 DEBUG:teuthology.provision.fog.smithi125:FOG request: DELETE /task/cancel {"id": 499527} 2023-09-21T12:30:53.384 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi097 2023-09-21T12:30:53.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-21T12:30:53.527 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7499cc7478c741ac98fd6b5b27ad587e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi097 2023-09-21T12:30:53.530 INFO:teuthology.nuke:Checking targets against current locks 2023-09-21T12:30:53.591 DEBUG:teuthology.nuke:shortname: smithi097 2023-09-21T12:30:53.592 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-21T12:30:53.615 DEBUG:teuthology.nuke:shortname: smithi125 2023-09-21T12:30:53.615 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-21T12:30:53.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399868', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:26:25.747689', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5PsAo33mTwEeo1Eu0EdUZ9KfA//4SDOy28XEJkXGfqNMeL3b7Qq8CbTO4CRaDEUE+6minPuzhmX0SCWe1ivyc='} 2023-09-21T12:30:53.643 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2023-09-20_16:32:45-fs-wip-lusov-config-format-distro-default-smithi/7399868', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 12:26:25.746193', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKrKaocHzvsx1FEGng3U74HTVChGLdJF4uALWtSJ6xbpJspvUuq2DiepEzmRIrn+LgzLzLIgo9uiaz6dsWDgbHk='} 2023-09-21T12:30:53.663 INFO:teuthology.orchestra.console.smithi097:Power off 2023-09-21T12:30:53.665 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:30:53.699 INFO:teuthology.orchestra.console.smithi125:Power off 2023-09-21T12:30:53.700 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:31:13.741 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:31:13.746 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:31:13.768 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:31:13.768 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:31:33.816 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:31:33.817 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:31:33.837 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:31:33.837 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T12:31:53.897 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:31:53.898 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:31:53.916 DEBUG:teuthology.orchestra.console.smithi125:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T12:31:53.917 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:31:54.908 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:31:54.924 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:31:58.910 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:31:58.925 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:00.020 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:00.034 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:04.026 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:04.035 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:05.135 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:05.145 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:09.138 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:09.146 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:10.248 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:10.255 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:14.254 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:14.262 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:15.365 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:15.372 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:19.370 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:19.378 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:20.481 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:20.487 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:24.486 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:24.493 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:25.596 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:25.603 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:29.598 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:29.606 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:30.708 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:30.714 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:34.714 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:34.721 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:35.824 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:35.829 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:39.830 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:39.838 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:40.940 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:40.945 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:44.946 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:44.953 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:46.056 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:46.064 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:50.058 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:50.065 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:51.166 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:51.175 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:32:55.170 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:55.177 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:32:56.280 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:32:56.288 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:33:00.285 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:33:00.294 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:33:01.397 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:33:01.402 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:33:05.402 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:33:05.410 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T12:33:06.513 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T12:33:06.514 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T12:33:06.515 ERROR:teuthology.orchestra.console.smithi097:Failed to power off 2023-09-21T12:33:06.521 DEBUG:teuthology.orchestra.console.smithi125:check power output: 2023-09-21T12:33:06.521 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T12:33:06.522 ERROR:teuthology.orchestra.console.smithi125:Failed to power off 2023-09-21T12:33:06.675 INFO:teuthology.lock.ops:unlocked: smithi097.front.sepia.ceph.com 2023-09-21T12:33:06.691 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com 2023-09-21T12:33:06.869 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi097