2023-11-04T03:25:46.909 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2
2023-11-04T03:25:46.911 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-04T03:25:47.088 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f3d9165eb20>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f3d90f0fd30>, signals=[15])
2023-11-04T03:25:47.089 INFO:teuthology.lock.ops:Start node 'smithi071.front.sepia.ceph.com' reimaging
2023-11-04T03:25:47.089 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: reset os type and version on server
2023-11-04T03:25:47.090 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server
2023-11-04T03:25:47.110 INFO:teuthology.lock.ops:Start node 'smithi076.front.sepia.ceph.com' reimaging
2023-11-04T03:25:47.113 INFO:teuthology.lock.ops:Updating [smithi076.front.sepia.ceph.com]: reset os type and version on server
2023-11-04T03:25:47.114 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server
2023-11-04T03:25:47.115 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /host {"name": "smithi071"}
2023-11-04T03:25:47.135 INFO:teuthology.lock.ops:Start node 'smithi122.front.sepia.ceph.com' reimaging
2023-11-04T03:25:47.136 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: reset os type and version on server
2023-11-04T03:25:47.136 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server
2023-11-04T03:25:47.138 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /host {"name": "smithi076"}
2023-11-04T03:25:47.160 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /host {"name": "smithi122"}
2023-11-04T03:25:47.187 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-11-04T03:25:47.199 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-11-04T03:25:47.231 DEBUG:teuthology.provision.fog.smithi071:Requesting image smithi_rhel_8.6 (ID 5)
2023-11-04T03:25:47.231 DEBUG:teuthology.provision.fog.smithi071:FOG request: PUT /host/176 {"imageID": 5}
2023-11-04T03:25:47.234 DEBUG:teuthology.provision.fog.smithi076:Requesting image smithi_rhel_8.6 (ID 5)
2023-11-04T03:25:47.235 DEBUG:teuthology.provision.fog.smithi076:FOG request: PUT /host/181 {"imageID": 5}
2023-11-04T03:25:47.236 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-11-04T03:25:47.283 INFO:teuthology.provision.fog.smithi071:Scheduling deploy of rhel 8.6
2023-11-04T03:25:47.283 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:25:47.289 DEBUG:teuthology.provision.fog.smithi122:Requesting image smithi_rhel_8.6 (ID 5)
2023-11-04T03:25:47.290 DEBUG:teuthology.provision.fog.smithi122:FOG request: PUT /host/227 {"imageID": 5}
2023-11-04T03:25:47.297 INFO:teuthology.provision.fog.smithi076:Scheduling deploy of rhel 8.6
2023-11-04T03:25:47.297 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:25:47.338 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of rhel 8.6
2023-11-04T03:25:47.338 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None
2023-11-04T03:25:47.351 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /tasktype {"name": "deploy"}
2023-11-04T03:25:47.355 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /tasktype {"name": "deploy"}
2023-11-04T03:25:47.387 DEBUG:teuthology.provision.fog.smithi076:FOG request: POST /host/181/task {"taskTypeID": 1}
2023-11-04T03:25:47.391 DEBUG:teuthology.provision.fog.smithi071:FOG request: POST /host/176/task {"taskTypeID": 1}
2023-11-04T03:25:47.395 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /tasktype {"name": "deploy"}
2023-11-04T03:25:47.426 DEBUG:teuthology.provision.fog.smithi122:FOG request: POST /host/227/task {"taskTypeID": 1}
2023-11-04T03:25:47.437 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:25:47.441 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:25:47.470 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None
2023-11-04T03:25:47.541 INFO:teuthology.orchestra.console.smithi071:Power off
2023-11-04T03:25:47.542 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:25:47.558 INFO:teuthology.orchestra.console.smithi076:Power off
2023-11-04T03:25:47.559 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:25:47.568 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off
2023-11-04T03:25:47.568 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:25:47.579 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off
2023-11-04T03:25:47.580 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:25:47.589 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off
2023-11-04T03:25:47.603 INFO:teuthology.orchestra.console.smithi122:Power off
2023-11-04T03:25:47.603 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:25:47.612 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off
2023-11-04T03:25:47.691 INFO:teuthology.orchestra.console.smithi071:Power off completed
2023-11-04T03:25:47.713 INFO:teuthology.orchestra.console.smithi076:Power off completed
2023-11-04T03:25:47.792 INFO:teuthology.orchestra.console.smithi071:Power on
2023-11-04T03:25:47.793 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T03:25:47.814 INFO:teuthology.orchestra.console.smithi076:Power on
2023-11-04T03:25:47.815 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T03:25:47.821 DEBUG:teuthology.orchestra.console.smithi071:power on output: Chassis Power Control: Up/On
2023-11-04T03:25:47.822 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:25:47.837 DEBUG:teuthology.orchestra.console.smithi076:power on output: Chassis Power Control: Up/On
2023-11-04T03:25:47.837 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:25:47.845 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off
2023-11-04T03:25:47.858 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off
2023-11-04T03:25:51.850 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:25:51.859 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:25:51.959 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off
2023-11-04T03:25:51.967 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off
2023-11-04T03:25:55.965 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:25:55.972 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:25:56.074 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on
2023-11-04T03:25:56.081 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on
2023-11-04T03:25:56.176 INFO:teuthology.orchestra.console.smithi071:Power on completed
2023-11-04T03:25:56.182 INFO:teuthology.orchestra.console.smithi076:Power on completed
2023-11-04T03:25:56.277 INFO:teuthology.provision.fog.smithi071:Waiting for deploy to finish
2023-11-04T03:25:56.278 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:25:56.283 INFO:teuthology.provision.fog.smithi076:Waiting for deploy to finish
2023-11-04T03:25:56.283 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:26:07.672 DEBUG:teuthology.orchestra.console.smithi122:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:26:07.672 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:26:11.363 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:26:11.366 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:26:26.450 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:26:26.451 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:26:27.740 DEBUG:teuthology.orchestra.console.smithi122:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:26:27.740 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:26:41.541 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:26:41.543 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:26:47.808 DEBUG:teuthology.orchestra.console.smithi122:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:26:47.808 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:26:48.818 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:26:52.822 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:26:53.930 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:26:56.633 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:26:56.634 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:26:57.933 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:26:59.042 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:03.046 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:04.155 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:08.158 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:09.268 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:11.711 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:27:11.715 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:27:13.271 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:14.381 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:18.386 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:19.495 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:23.498 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:24.607 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:26.794 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:27:26.798 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:27:28.610 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:29.719 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:33.723 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:34.832 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:38.834 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:39.942 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:41.872 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:27:41.874 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:27:43.946 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:45.054 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:49.058 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:50.166 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:54.170 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:27:55.278 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:27:56.961 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:27:56.965 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:27:59.282 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:28:00.391 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:28:00.392 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T03:28:00.392 ERROR:teuthology.orchestra.console.smithi122:Failed to power off
2023-11-04T03:28:00.392 INFO:teuthology.orchestra.console.smithi122:Power on
2023-11-04T03:28:00.393 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T03:28:12.074 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:28:12.077 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:28:20.460 DEBUG:teuthology.orchestra.console.smithi122:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:28:20.460 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T03:28:27.170 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:28:27.172 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None
2023-11-04T03:28:27.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:28:30.334 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:28:36.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:28:39.418 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:28:40.528 DEBUG:teuthology.orchestra.console.smithi122:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:28:40.528 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-04T03:28:42.270 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:28:45.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:28:48.474 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:28:54.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:28:57.354 DEBUG:teuthology.provision.fog.smithi076:FOG request: GET /task/active None
2023-11-04T03:28:57.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:28:57.530 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:29:00.506 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:29:00.600 DEBUG:teuthology.orchestra.console.smithi122:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:29:00.600 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:01.608 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:03.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:05.610 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:06.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:06.586 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:29:06.718 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:09.566 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:29:10.720 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:11.829 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:12.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:15.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:15.642 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:29:15.831 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:16.940 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:18.650 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:29:20.944 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:21.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:22.053 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:24.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:24.698 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:29:26.055 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:27.165 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:27.706 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:29:30.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:31.167 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:32.275 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:33.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:36.278 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:36.762 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:29:37.387 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:41.390 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:42.501 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:42.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:45.818 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:29:46.504 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:47.612 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:51.614 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:51.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:29:52.723 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:29:54.878 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:29:56.726 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:29:57.834 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:30:00.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:30:01.836 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:30:02.946 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:30:06.950 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:30:08.060 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:30:12.062 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:30:13.170 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:30:13.171 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T03:30:13.171 DEBUG:teuthology.provision.fog.smithi122:Canceling deploy task with ID 582177
2023-11-04T03:30:13.171 DEBUG:teuthology.provision.fog.smithi122:FOG request: DELETE /task/cancel {"id": 582177}
2023-11-04T03:30:13.308 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi122
2023-11-04T03:30:13.309 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-04T03:30:13.443 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=392757cd6d38429ca5746611105193a6
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi122
2023-11-04T03:30:13.445 INFO:teuthology.nuke:Checking targets against current locks
2023-11-04T03:30:13.509 DEBUG:teuthology.nuke:shortname: smithi071
2023-11-04T03:30:13.509 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-04T03:30:13.539 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445863', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 03:25:46.037217', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP7cskD1IbbaefcV4w9XDFmW5tuft5nBC7VDAlJfH69w55tCWSlC1usRoIa2/2FHMoFGtV8hPEvrTs+BiZu++So='}
2023-11-04T03:30:13.545 DEBUG:teuthology.nuke:shortname: smithi076
2023-11-04T03:30:13.546 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-04T03:30:13.578 DEBUG:teuthology.nuke:shortname: smithi122
2023-11-04T03:30:13.579 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-04T03:30:13.580 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445863', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 03:25:46.035441', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAoF8qyotyJn0qMd8a9t8C3A3sHInTIruwOg7MD6/Ktc7Pr8JqMkxBg3AK/uNH91QAZcy1hxA0o9vb8Utf4SIeU='}
2023-11-04T03:30:13.606 INFO:teuthology.orchestra.console.smithi071:Power off
2023-11-04T03:30:13.607 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:30:13.618 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-03_16:49:22-fs-rishabh-unqiue-fsnames-testing-default-smithi/7445863', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-04 03:25:46.033471', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPGRmeEJRUESrG+o67eqmISpmBPThJb8jCynsT0aDZ7CRiRME6fa0oXbEyLiOG2uFrbXOas19o08kxxfT2uQonQ='}
2023-11-04T03:30:13.651 INFO:teuthology.orchestra.console.smithi076:Power off
2023-11-04T03:30:13.652 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:30:13.662 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off
2023-11-04T03:30:13.663 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:30:13.676 INFO:teuthology.orchestra.console.smithi122:Power off
2023-11-04T03:30:13.678 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:30:13.689 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off
2023-11-04T03:30:13.689 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:30:13.697 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on
2023-11-04T03:30:13.714 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on
2023-11-04T03:30:17.702 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:30:17.716 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:30:17.812 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on
2023-11-04T03:30:17.826 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off
2023-11-04T03:30:17.927 INFO:teuthology.orchestra.console.smithi076:Power off completed
2023-11-04T03:30:18.077 INFO:teuthology.lock.ops:unlocked: smithi076.front.sepia.ceph.com
2023-11-04T03:30:21.818 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:30:21.927 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off
2023-11-04T03:30:22.028 INFO:teuthology.orchestra.console.smithi071:Power off completed
2023-11-04T03:30:22.177 INFO:teuthology.lock.ops:unlocked: smithi071.front.sepia.ceph.com
2023-11-04T03:30:30.710 WARNING:teuthology.provision.fog.smithi071:timed out
2023-11-04T03:30:33.751 DEBUG:teuthology.orchestra.console.smithi122:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:30:33.751 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:30:36.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:30:53.821 DEBUG:teuthology.orchestra.console.smithi122:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:30:53.821 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-04T03:30:55.227 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:31:00.890 WARNING:teuthology.provision.fog.smithi076:timed out
2023-11-04T03:31:01.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:04.282 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:31:06.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:09.950 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:31:10.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:13.338 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:31:13.888 DEBUG:teuthology.orchestra.console.smithi122:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-11-04T03:31:13.888 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:14.897 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:15.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:18.899 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:19.034 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:31:19.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:20.008 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:22.398 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:31:24.010 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:25.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:25.120 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:28.090 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:31:28.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:29.122 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:30.233 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:31.455 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:31:34.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:34.235 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:35.348 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:37.146 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:31:37.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:39.350 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:40.461 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:40.538 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:31:43.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:44.464 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:45.574 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:46.202 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:31:46.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:49.578 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:49.594 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:31:50.687 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:52.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:54.690 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:31:55.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:31:55.803 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:31:59.806 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:32:00.914 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:32:04.918 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:32:06.028 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:32:10.030 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:32:10.746 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:32:11.139 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:32:14.074 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:32:15.141 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:32:16.250 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:32:16.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:32:19.802 WARNING:teuthology.provision.fog.smithi076:[Errno None] Unable to connect to port 22 on 172.21.15.76
2023-11-04T03:32:20.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:32:20.252 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:32:21.362 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:32:23.130 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71
2023-11-04T03:32:25.366 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-04T03:32:25.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-04T03:32:25.928 ERROR:teuthology.orchestra.connection:Error authenticating with smithi076.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-04T03:32:26.475 DEBUG:teuthology.orchestra.console.smithi122:check power output: 
2023-11-04T03:32:26.476 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-11-04T03:32:26.476 ERROR:teuthology.orchestra.console.smithi122:Failed to power off
2023-11-04T03:32:26.633 INFO:teuthology.lock.ops:unlocked: smithi122.front.sepia.ceph.com
2023-11-04T03:32:28.464 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi122