2023-09-21T14:52:51.827 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc
2023-09-21T14:52:51.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T14:52:52.009 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7feaca7f2d90>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7feacb2e5670>, signals=[15])
2023-09-21T14:52:52.010 INFO:teuthology.lock.ops:Start node 'smithi005.front.sepia.ceph.com' reimaging
2023-09-21T14:52:52.011 INFO:teuthology.lock.ops:Updating [smithi005.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T14:52:52.011 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server
2023-09-21T14:52:52.042 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging
2023-09-21T14:52:52.042 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T14:52:52.043 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server
2023-09-21T14:52:52.044 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /host {"name": "smithi005"}
2023-09-21T14:52:52.066 INFO:teuthology.lock.ops:Start node 'smithi190.front.sepia.ceph.com' reimaging
2023-09-21T14:52:52.067 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: reset os type and version on server
2023-09-21T14:52:52.067 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server
2023-09-21T14:52:52.068 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /host {"name": "smithi118"}
2023-09-21T14:52:52.092 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /host {"name": "smithi190"}
2023-09-21T14:52:52.117 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-09-21T14:52:52.132 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-09-21T14:52:52.155 DEBUG:teuthology.provision.fog.smithi005:Requesting image smithi_centos_8.stream (ID 3)
2023-09-21T14:52:52.156 DEBUG:teuthology.provision.fog.smithi005:FOG request: PUT /host/111 {"imageID": 3}
2023-09-21T14:52:52.159 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-09-21T14:52:52.180 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_centos_8.stream (ID 3)
2023-09-21T14:52:52.180 DEBUG:teuthology.provision.fog.smithi118:FOG request: PUT /host/223 {"imageID": 3}
2023-09-21T14:52:52.201 DEBUG:teuthology.provision.fog.smithi190:Requesting image smithi_centos_8.stream (ID 3)
2023-09-21T14:52:52.201 DEBUG:teuthology.provision.fog.smithi190:FOG request: PUT /host/295 {"imageID": 3}
2023-09-21T14:52:52.207 INFO:teuthology.provision.fog.smithi005:Scheduling deploy of centos 8.stream
2023-09-21T14:52:52.208 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:52:52.230 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of centos 8.stream
2023-09-21T14:52:52.230 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None
2023-09-21T14:52:52.256 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of centos 8.stream
2023-09-21T14:52:52.256 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:52:52.312 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T14:52:52.318 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T14:52:52.351 DEBUG:teuthology.provision.fog.smithi005:FOG request: POST /host/111/task {"taskTypeID": 1}
2023-09-21T14:52:52.357 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /tasktype {"name": "deploy"}
2023-09-21T14:52:52.358 DEBUG:teuthology.provision.fog.smithi118:FOG request: POST /host/223/task {"taskTypeID": 1}
2023-09-21T14:52:52.405 DEBUG:teuthology.provision.fog.smithi190:FOG request: POST /host/295/task {"taskTypeID": 1}
2023-09-21T14:52:52.419 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None
2023-09-21T14:52:52.426 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:52:52.456 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:52:52.566 INFO:teuthology.orchestra.console.smithi005:Power off
2023-09-21T14:52:52.567 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:52:52.580 INFO:teuthology.orchestra.console.smithi190:Power off
2023-09-21T14:52:52.581 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:52:52.591 INFO:teuthology.orchestra.console.smithi118:Power off
2023-09-21T14:52:52.592 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:52:52.599 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off
2023-09-21T14:52:52.600 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:52:52.607 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off
2023-09-21T14:52:52.607 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:52:52.620 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-09-21T14:52:52.626 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2023-09-21T14:52:52.727 INFO:teuthology.orchestra.console.smithi190:Power off completed
2023-09-21T14:52:52.828 INFO:teuthology.orchestra.console.smithi190:Power on
2023-09-21T14:52:52.828 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T14:52:52.849 DEBUG:teuthology.orchestra.console.smithi190:power on output: Chassis Power Control: Up/On
2023-09-21T14:52:52.850 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:52:52.872 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2023-09-21T14:52:56.622 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:52:56.732 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-09-21T14:52:56.873 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:52:56.982 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on
2023-09-21T14:52:57.083 INFO:teuthology.orchestra.console.smithi190:Power on completed
2023-09-21T14:52:57.184 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish
2023-09-21T14:52:57.184 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:53:00.734 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:53:00.842 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off
2023-09-21T14:53:00.943 INFO:teuthology.orchestra.console.smithi005:Power off completed
2023-09-21T14:53:01.044 INFO:teuthology.orchestra.console.smithi005:Power on
2023-09-21T14:53:01.045 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T14:53:01.067 DEBUG:teuthology.orchestra.console.smithi005:power on output: Chassis Power Control: Up/On
2023-09-21T14:53:01.068 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:53:01.093 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off
2023-09-21T14:53:05.098 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:53:05.207 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off
2023-09-21T14:53:09.210 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:53:09.411 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-09-21T14:53:09.512 INFO:teuthology.orchestra.console.smithi005:Power on completed
2023-09-21T14:53:09.613 INFO:teuthology.provision.fog.smithi005:Waiting for deploy to finish
2023-09-21T14:53:09.613 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:53:12.302 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:53:12.660 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:53:12.660 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:53:24.718 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:53:27.402 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:53:32.733 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:53:32.733 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:53:39.830 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:53:42.497 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:53:52.797 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:53:52.798 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:53:53.807 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:53:54.908 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:53:57.593 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:53:57.809 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:53:58.919 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:02.922 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:04.031 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:08.034 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:09.143 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:09.992 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:54:12.674 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:54:13.145 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:14.254 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:18.258 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:19.367 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:23.372 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:24.481 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:25.087 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:54:27.774 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:54:28.483 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:29.592 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:33.595 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:34.704 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:38.708 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:39.817 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:40.166 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:54:42.863 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:54:43.819 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:44.930 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:48.934 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:50.044 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:54.050 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:54:55.160 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:54:55.256 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:54:57.946 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:54:59.162 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:55:00.270 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:55:04.274 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:55:05.386 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:55:05.387 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T14:55:05.388 ERROR:teuthology.orchestra.console.smithi118:Failed to power off
2023-09-21T14:55:05.388 INFO:teuthology.orchestra.console.smithi118:Power on
2023-09-21T14:55:05.388 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T14:55:10.366 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:55:13.030 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:55:25.456 DEBUG:teuthology.orchestra.console.smithi118:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:55:25.456 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T14:55:25.463 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:55:28.130 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:55:40.562 DEBUG:teuthology.provision.fog.smithi005:FOG request: GET /task/active None
2023-09-21T14:55:40.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:55:43.250 DEBUG:teuthology.provision.fog.smithi190:FOG request: GET /task/active None
2023-09-21T14:55:43.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:55:45.521 DEBUG:teuthology.orchestra.console.smithi118:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:55:45.522 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-09-21T14:55:59.131 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:56:01.690 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:56:05.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:56:05.593 DEBUG:teuthology.orchestra.console.smithi118:power on output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:56:05.594 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:06.603 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:07.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:56:08.186 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:56:10.606 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:10.746 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:56:11.715 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:14.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:56:15.718 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:16.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:56:16.826 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:17.242 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:56:19.802 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:56:20.829 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:21.937 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:23.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:56:25.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:56:25.939 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:27.047 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:31.050 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:32.159 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:36.162 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:37.270 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:41.274 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:42.383 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:46.386 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:47.496 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:51.498 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:52.606 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:56:56.611 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:56:57.720 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:57:01.722 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:02.831 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:57:06.836 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:07.945 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:57:11.950 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:13.066 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:57:17.070 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:18.178 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:57:18.179 WARNING:teuthology.contextutil:'wait for power on' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T14:57:18.179 DEBUG:teuthology.provision.fog.smithi118:Canceling deploy task with ID 500132
2023-09-21T14:57:18.180 DEBUG:teuthology.provision.fog.smithi118:FOG request: DELETE /task/cancel {"id": 500132}
2023-09-21T14:57:18.249 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi118
2023-09-21T14:57:18.250 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-09-21T14:57:19.496 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b7261242eff8428891fa8515d48a0174
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi118
2023-09-21T14:57:19.551 INFO:teuthology.nuke:Checking targets against current locks
2023-09-21T14:57:19.638 DEBUG:teuthology.nuke:shortname: smithi005
2023-09-21T14:57:19.638 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T14:57:19.675 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/svelar-2023-09-20_07:59:56-rados-wip-svelar-testing-1-2023-09-20-1125-distro-default-smithi/7398874', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 14:52:50.988634', 'locked_by': 'scheduled_svelar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKzQVhAYcb8GM1KrHbO8w3M4Z1RG68snsNkyZPXgIIHQYmnozRV3ID7Hpge88cxnFzzYqf2kCyI5g6StiRiEO8I='}
2023-09-21T14:57:19.676 DEBUG:teuthology.nuke:shortname: smithi118
2023-09-21T14:57:19.676 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T14:57:19.698 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/svelar-2023-09-20_07:59:56-rados-wip-svelar-testing-1-2023-09-20-1125-distro-default-smithi/7398874', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 14:52:50.985065', 'locked_by': 'scheduled_svelar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHRE42EXVWAQ9kqDjOQ6hOHfXJ+0+i8Bi+Rl2xk80yQLZTfeiJa0sLUT0v55bz/u7u8H3agZr2OeOhE92veP/Jk='}
2023-09-21T14:57:19.703 DEBUG:teuthology.nuke:shortname: smithi190
2023-09-21T14:57:19.703 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-09-21T14:57:19.768 INFO:teuthology.orchestra.console.smithi005:Power off
2023-09-21T14:57:19.769 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:57:19.781 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/svelar-2023-09-20_07:59:56-rados-wip-svelar-testing-1-2023-09-20-1125-distro-default-smithi/7398874', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-21 14:52:50.986946', 'locked_by': 'scheduled_svelar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKjcoTae50oIEhxUJYiNYScPwRtkq3V2DSL8adoGXaPI9ys9/4IOpAvZgqxvikaFo57v5xKioFRsaDNcGAa5KCQ='}
2023-09-21T14:57:19.817 INFO:teuthology.orchestra.console.smithi118:Power off
2023-09-21T14:57:19.818 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:57:19.855 INFO:teuthology.orchestra.console.smithi190:Power off
2023-09-21T14:57:19.855 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:57:19.865 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off
2023-09-21T14:57:19.866 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:19.878 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off
2023-09-21T14:57:19.879 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:19.897 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-09-21T14:57:19.901 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on
2023-09-21T14:57:24.193 WARNING:teuthology.provision.fog.smithi005:timed out
2023-09-21T14:57:24.289 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:24.297 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:24.400 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on
2023-09-21T14:57:24.408 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on
2023-09-21T14:57:25.810 WARNING:teuthology.provision.fog.smithi190:timed out
2023-09-21T14:57:28.410 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:28.418 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:57:28.520 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off
2023-09-21T14:57:28.526 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off
2023-09-21T14:57:28.623 INFO:teuthology.orchestra.console.smithi005:Power off completed
2023-09-21T14:57:28.632 INFO:teuthology.orchestra.console.smithi190:Power off completed
2023-09-21T14:57:28.783 INFO:teuthology.lock.ops:unlocked: smithi190.front.sepia.ceph.com
2023-09-21T14:57:28.792 INFO:teuthology.lock.ops:unlocked: smithi005.front.sepia.ceph.com
2023-09-21T14:57:30.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:57:31.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:57:39.973 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:57:39.974 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:57:48.703 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:57:50.238 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:57:54.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:57:56.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:57:57.786 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:57:59.326 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:58:00.044 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:58:00.044 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-09-21T14:58:03.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:05.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:06.846 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:58:08.410 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:58:12.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:14.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:15.934 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:58:17.466 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:58:20.109 DEBUG:teuthology.orchestra.console.smithi118:power off output: Error: Unable to establish IPMI v2 / RMCP+ session
2023-09-21T14:58:20.110 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:58:21.119 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:58:21.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:23.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:25.023 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:58:25.120 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:58:26.235 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:58:26.522 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:58:30.241 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:58:31.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:31.352 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:58:32.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:34.106 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:58:35.355 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:58:35.582 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:58:36.465 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:58:40.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:40.467 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:58:41.578 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:58:41.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:43.162 WARNING:teuthology.provision.fog.smithi005:[Errno None] Unable to connect to port 22 on 172.21.15.5
2023-09-21T14:58:44.670 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:58:45.581 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:58:46.708 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:58:49.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:50.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:58:50.710 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:58:51.821 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:58:53.754 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:58:55.825 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:58:56.936 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:58:59.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:59:00.941 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:59:02.050 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:59:06.054 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:59:07.164 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:59:11.166 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:59:12.275 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:59:16.278 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:59:17.388 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:59:18.298 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:59:21.390 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:59:22.500 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:59:24.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:59:26.504 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:59:26.916 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190
2023-09-21T14:59:27.617 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:59:31.622 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-09-21T14:59:32.731 DEBUG:teuthology.orchestra.console.smithi118:check power output: 
2023-09-21T14:59:32.732 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds
2023-09-21T14:59:32.733 ERROR:teuthology.orchestra.console.smithi118:Failed to power off
2023-09-21T14:59:32.891 INFO:teuthology.lock.ops:unlocked: smithi118.front.sepia.ceph.com
2023-09-21T14:59:32.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-09-21T14:59:33.041 ERROR:teuthology.orchestra.connection:Error authenticating with smithi190.front.sepia.ceph.com: Authentication failed.
2023-09-21T14:59:33.048 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-09-21T14:59:33.049 DEBUG:teuthology.orchestra.run.smithi190:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-09-21T14:59:33.051 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-09-21T14:59:33.203 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 85, in create
    self.remote.console.power_on()
  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/console.py", line 252, in power_on
    raise RuntimeError(err_msg)
RuntimeError: Failed to power on smithi118