2023-09-21T14:17:58.847 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-21T14:17:58.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-21T14:17:58.981 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f30dbeb4670>, signals=[15]) 2023-09-21T14:17:58.982 INFO:teuthology.lock.ops:Start node 'smithi097.front.sepia.ceph.com' reimaging 2023-09-21T14:17:58.982 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: reset os type and version on server 2023-09-21T14:17:58.983 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-09-21T14:17:59.006 INFO:teuthology.lock.ops:Start node 'smithi105.front.sepia.ceph.com' reimaging 2023-09-21T14:17:59.006 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: reset os type and version on server 2023-09-21T14:17:59.007 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-09-21T14:17:59.008 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /host {"name": "smithi097"} 2023-09-21T14:17:59.030 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /host {"name": "smithi105"} 2023-09-21T14:17:59.082 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-09-21T14:17:59.097 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-09-21T14:17:59.123 DEBUG:teuthology.provision.fog.smithi097:Requesting image smithi_centos_9.stream (ID 10) 2023-09-21T14:17:59.123 DEBUG:teuthology.provision.fog.smithi097:FOG request: PUT /host/202 {"imageID": 10} 2023-09-21T14:17:59.139 DEBUG:teuthology.provision.fog.smithi105:Requesting image smithi_centos_9.stream (ID 10) 2023-09-21T14:17:59.139 DEBUG:teuthology.provision.fog.smithi105:FOG request: PUT /host/210 {"imageID": 10} 2023-09-21T14:17:59.166 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of centos 9.stream 2023-09-21T14:17:59.167 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-09-21T14:17:59.188 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of centos 9.stream 2023-09-21T14:17:59.188 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-09-21T14:17:59.248 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /tasktype {"name": "deploy"} 2023-09-21T14:17:59.254 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /tasktype {"name": "deploy"} 2023-09-21T14:17:59.291 DEBUG:teuthology.provision.fog.smithi097:FOG request: POST /host/202/task {"taskTypeID": 1} 2023-09-21T14:17:59.301 DEBUG:teuthology.provision.fog.smithi105:FOG request: POST /host/210/task {"taskTypeID": 1} 2023-09-21T14:17:59.350 DEBUG:teuthology.provision.fog.smithi097:FOG request: GET /task/active None 2023-09-21T14:17:59.356 DEBUG:teuthology.provision.fog.smithi105:FOG request: GET /task/active None 2023-09-21T14:17:59.452 INFO:teuthology.orchestra.console.smithi097:Power off 2023-09-21T14:17:59.453 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:17:59.467 INFO:teuthology.orchestra.console.smithi105:Power off 2023-09-21T14:17:59.468 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:18:19.525 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:18:19.526 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:18:19.539 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:18:19.540 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:18:39.597 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:18:39.598 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:18:39.608 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:18:39.608 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:18:59.668 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:18:59.669 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:18:59.679 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:18:59.680 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:00.679 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:00.689 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:04.682 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:04.690 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:05.792 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:05.798 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:09.794 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:09.801 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:10.904 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:10.911 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:14.910 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:14.917 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:16.019 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:16.025 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:20.022 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:20.028 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:21.131 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:21.137 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:25.134 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:25.142 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:26.245 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:26.251 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:30.250 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:30.257 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:31.361 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:31.365 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:35.365 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:35.372 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:36.475 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:36.482 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:40.478 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:40.485 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:41.588 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:41.593 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:45.593 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:45.600 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:46.703 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:46.708 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:50.706 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:50.713 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:51.816 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:51.820 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:19:55.818 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:55.826 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:19:56.929 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:19:56.934 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:20:00.934 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:20:00.941 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:20:02.045 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:20:02.050 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:20:06.050 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:20:06.056 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:20:07.159 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:20:07.165 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:20:11.162 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:20:11.172 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:20:12.274 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:20:12.275 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T14:20:12.276 ERROR:teuthology.orchestra.console.smithi097:Failed to power off 2023-09-21T14:20:12.276 INFO:teuthology.orchestra.console.smithi097:Power on 2023-09-21T14:20:12.276 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T14:20:12.284 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:20:12.285 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T14:20:12.285 ERROR:teuthology.orchestra.console.smithi105:Failed to power off 2023-09-21T14:20:12.285 INFO:teuthology.orchestra.console.smithi105:Power on 2023-09-21T14:20:12.285 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T14:20:32.345 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:20:32.346 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T14:20:32.356 DEBUG:teuthology.orchestra.console.smithi105:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:20:32.356 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T14:20:52.417 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:20:52.419 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T14:20:52.427 DEBUG:teuthology.orchestra.console.smithi105:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:20:52.428 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-21T14:21:12.489 DEBUG:teuthology.orchestra.console.smithi097:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:21:12.490 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:12.499 DEBUG:teuthology.orchestra.console.smithi105:power on output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:21:12.500 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:13.500 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:13.507 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:17.503 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:17.510 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:18.613 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:18.619 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:22.618 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:22.628 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:23.730 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:23.737 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:27.735 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:27.743 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:28.846 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:28.851 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:32.850 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:32.857 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:33.959 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:33.965 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:37.965 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:37.973 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:39.076 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:39.081 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:43.078 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:43.084 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:44.186 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:44.194 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:48.190 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:48.198 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:49.301 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:49.307 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:53.306 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:53.313 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:54.416 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:54.421 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:21:58.418 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:58.425 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:21:59.528 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:21:59.532 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:22:03.532 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:03.539 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:04.642 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:22:04.646 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:22:08.646 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:08.652 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:09.755 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:22:09.760 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:22:13.758 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:13.765 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:14.868 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:22:14.874 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:22:18.873 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:18.881 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:19.984 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:22:19.989 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:22:23.986 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:23.993 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:22:25.095 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:22:25.096 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T14:22:25.096 DEBUG:teuthology.provision.fog.smithi097:Canceling deploy task with ID 499985 2023-09-21T14:22:25.096 DEBUG:teuthology.provision.fog.smithi097:FOG request: DELETE /task/cancel {"id": 499985} 2023-09-21T14:22:25.101 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:22:25.102 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T14:22:25.102 DEBUG:teuthology.provision.fog.smithi105:Canceling deploy task with ID 499986 2023-09-21T14:22:25.102 DEBUG:teuthology.provision.fog.smithi105:FOG request: DELETE /task/cancel {"id": 499986} 2023-09-21T14:22:25.156 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi105 2023-09-21T14:22:25.157 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-21T14:22:25.305 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=76fe95437b194b51be35e7b855cc5024 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi105 2023-09-21T14:22:25.308 INFO:teuthology.nuke:Checking targets against current locks 2023-09-21T14:22:25.368 DEBUG:teuthology.nuke:shortname: smithi097 2023-09-21T14:22:25.369 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-21T14:22:25.393 DEBUG:teuthology.nuke:shortname: smithi105 2023-09-21T14:22:25.393 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-21T14:22:25.396 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/svelar-2023-09-20_07:59:56-rados-wip-svelar-testing-1-2023-09-20-1125-distro-default-smithi/7398791', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 14:17:58.037301', 'locked_by': 'scheduled_svelar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5PsAo33mTwEeo1Eu0EdUZ9KfA//4SDOy28XEJkXGfqNMeL3b7Qq8CbTO4CRaDEUE+6minPuzhmX0SCWe1ivyc='} 2023-09-21T14:22:25.414 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/svelar-2023-09-20_07:59:56-rados-wip-svelar-testing-1-2023-09-20-1125-distro-default-smithi/7398791', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 14:17:58.039105', 'locked_by': 'scheduled_svelar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJptX02Xv1cnuL+0u1sr/6lwByx6nivuBRCcqhvjH1/U0Rf2g+qKaUsMSwf+FRz6zuwx5VHORTRpsfjGGHXK4a8='} 2023-09-21T14:22:25.445 INFO:teuthology.orchestra.console.smithi097:Power off 2023-09-21T14:22:25.445 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:22:25.475 INFO:teuthology.orchestra.console.smithi105:Power off 2023-09-21T14:22:25.476 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:22:45.513 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:22:45.514 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:22:45.544 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:22:45.544 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:23:05.586 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:23:05.587 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:23:05.612 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:23:05.612 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-21T14:23:25.657 DEBUG:teuthology.orchestra.console.smithi097:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:23:25.658 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:25.679 DEBUG:teuthology.orchestra.console.smithi105:power off output: Error: Unable to establish IPMI v2 / RMCP+ session 2023-09-21T14:23:25.680 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:26.668 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:23:26.690 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:23:30.670 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:30.692 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:31.781 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:23:31.801 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:23:35.786 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:35.802 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:36.896 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:23:36.911 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:23:40.898 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:40.912 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:42.009 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:23:42.022 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:23:46.014 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:46.023 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:47.125 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:23:47.132 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:23:51.130 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:51.138 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:52.241 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:23:52.246 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:23:56.246 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:56.253 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:23:57.356 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:23:57.361 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:01.362 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:01.370 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:02.472 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:24:02.479 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:06.474 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:06.481 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:07.584 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:24:07.590 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:11.589 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:11.597 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:12.700 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:24:12.706 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:16.706 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:16.716 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:17.818 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:24:17.824 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:21.822 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:21.831 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:22.934 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:24:22.941 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:26.938 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:26.946 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:28.048 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:24:28.054 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:32.054 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:32.062 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:33.165 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:24:33.171 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:37.170 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:37.178 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-21T14:24:38.281 DEBUG:teuthology.orchestra.console.smithi097:check power output: 2023-09-21T14:24:38.281 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T14:24:38.282 ERROR:teuthology.orchestra.console.smithi097:Failed to power off 2023-09-21T14:24:38.287 DEBUG:teuthology.orchestra.console.smithi105:check power output: 2023-09-21T14:24:38.287 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-09-21T14:24:38.287 ERROR:teuthology.orchestra.console.smithi105:Failed to power off 2023-09-21T14:24:38.442 INFO:teuthology.lock.ops:unlocked: smithi097.front.sepia.ceph.com 2023-09-21T14:24:38.451 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com 2023-09-21T14:24:38.685 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create self.remote.console.power_on() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on raise RuntimeError(err_msg) RuntimeError: Failed to power on smithi105