2024-04-06T16:10:18.958 INFO:root:teuthology version: 0.0.1.dev267+g6c63784
2024-04-06T16:10:18.959 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-06T16:10:19.193 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f9135f94f70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f9136f35c10>, signals=[15])
2024-04-06T16:10:19.194 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging
2024-04-06T16:10:19.194 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server
2024-04-06T16:10:19.194 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server
2024-04-06T16:10:19.210 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging
2024-04-06T16:10:19.211 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server
2024-04-06T16:10:19.211 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server
2024-04-06T16:10:19.226 INFO:teuthology.lock.ops:Start node 'smithi113.front.sepia.ceph.com' reimaging
2024-04-06T16:10:19.227 INFO:teuthology.lock.ops:Updating [smithi113.front.sepia.ceph.com]: reset os type and version on server
2024-04-06T16:10:19.227 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server
2024-04-06T16:10:19.252 INFO:teuthology.lock.ops:Start node 'smithi183.front.sepia.ceph.com' reimaging
2024-04-06T16:10:19.253 INFO:teuthology.lock.ops:Updating [smithi183.front.sepia.ceph.com]: reset os type and version on server
2024-04-06T16:10:19.253 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server
2024-04-06T16:10:19.326 DEBUG:teuthology.provision.fog.smithi050:Requesting image smithi_centos_9.stream (ID 10)
2024-04-06T16:10:19.345 DEBUG:teuthology.provision.fog.smithi062:Requesting image smithi_centos_9.stream (ID 10)
2024-04-06T16:10:19.373 DEBUG:teuthology.provision.fog.smithi183:Requesting image smithi_centos_9.stream (ID 10)
2024-04-06T16:10:19.377 DEBUG:teuthology.provision.fog.smithi113:Requesting image smithi_centos_9.stream (ID 10)
2024-04-06T16:10:19.387 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 9.stream
2024-04-06T16:10:19.401 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of centos 9.stream
2024-04-06T16:10:19.416 INFO:teuthology.provision.fog.smithi183:Scheduling deploy of centos 9.stream
2024-04-06T16:10:19.421 INFO:teuthology.provision.fog.smithi113:Scheduling deploy of centos 9.stream
2024-04-06T16:10:19.486 DEBUG:teuthology.provision.fog.smithi183:Canceling deploy task with ID 929481
2024-04-06T16:10:19.680 INFO:teuthology.orchestra.console.smithi113:Power off
2024-04-06T16:10:19.680 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-06T16:10:19.697 INFO:teuthology.orchestra.console.smithi062:Power off
2024-04-06T16:10:19.698 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-06T16:10:19.711 INFO:teuthology.orchestra.console.smithi050:Power off
2024-04-06T16:10:19.712 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-06T16:10:19.725 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off
2024-04-06T16:10:19.725 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:19.733 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off
2024-04-06T16:10:19.733 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:19.741 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off
2024-04-06T16:10:19.742 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:19.750 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on
2024-04-06T16:10:19.756 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-04-06T16:10:19.768 INFO:teuthology.orchestra.console.smithi183:Power off
2024-04-06T16:10:19.769 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-06T16:10:19.777 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-06T16:10:19.790 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off
2024-04-06T16:10:19.790 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:19.812 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on
2024-04-06T16:10:23.754 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:23.762 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:23.779 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:23.812 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:23.864 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on
2024-04-06T16:10:23.874 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-04-06T16:10:23.888 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-06T16:10:23.922 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on
2024-04-06T16:10:27.866 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:27.875 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:27.890 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:27.924 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:27.978 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off
2024-04-06T16:10:27.984 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-04-06T16:10:28.000 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2024-04-06T16:10:28.033 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on
2024-04-06T16:10:28.079 INFO:teuthology.orchestra.console.smithi113:Power off completed
2024-04-06T16:10:28.100 INFO:teuthology.orchestra.console.smithi050:Power off completed
2024-04-06T16:10:28.180 INFO:teuthology.orchestra.console.smithi113:Power on
2024-04-06T16:10:28.180 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-06T16:10:28.201 INFO:teuthology.orchestra.console.smithi050:Power on
2024-04-06T16:10:28.202 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-06T16:10:28.212 DEBUG:teuthology.orchestra.console.smithi113:power on output: Chassis Power Control: Up/On
2024-04-06T16:10:28.212 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:28.231 DEBUG:teuthology.orchestra.console.smithi050:power on output: Chassis Power Control: Up/On
2024-04-06T16:10:28.232 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:28.241 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off
2024-04-06T16:10:28.255 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2024-04-06T16:10:31.989 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:32.035 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:32.099 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-04-06T16:10:32.144 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2024-04-06T16:10:32.201 INFO:teuthology.orchestra.console.smithi062:Power off completed
2024-04-06T16:10:32.243 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:32.250 INFO:teuthology.orchestra.console.smithi183:Power off completed
2024-04-06T16:10:32.255 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:32.302 INFO:teuthology.orchestra.console.smithi062:Power on
2024-04-06T16:10:32.303 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-06T16:10:32.324 DEBUG:teuthology.orchestra.console.smithi062:power on output: Chassis Power Control: Up/On
2024-04-06T16:10:32.325 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:32.345 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-04-06T16:10:32.351 INFO:teuthology.orchestra.console.smithi183:Power on
2024-04-06T16:10:32.351 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-06T16:10:32.360 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off
2024-04-06T16:10:32.364 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2024-04-06T16:10:32.373 DEBUG:teuthology.orchestra.console.smithi183:power on output: Chassis Power Control: Up/On
2024-04-06T16:10:32.374 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:32.397 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2024-04-06T16:10:36.350 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:36.362 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:36.369 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:36.398 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:36.460 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-04-06T16:10:36.478 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-06T16:10:36.486 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on
2024-04-06T16:10:36.507 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2024-04-06T16:10:36.561 INFO:teuthology.orchestra.console.smithi062:Power on completed
2024-04-06T16:10:36.580 INFO:teuthology.orchestra.console.smithi050:Power on completed
2024-04-06T16:10:36.587 INFO:teuthology.orchestra.console.smithi113:Power on completed
2024-04-06T16:10:36.663 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish
2024-04-06T16:10:36.681 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish
2024-04-06T16:10:36.688 INFO:teuthology.provision.fog.smithi113:Waiting for deploy to finish
2024-04-06T16:10:40.510 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:10:40.618 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on
2024-04-06T16:10:40.719 INFO:teuthology.orchestra.console.smithi183:Power on completed
2024-04-06T16:10:40.820 INFO:teuthology.provision.fog.smithi183:Waiting for deploy to finish
2024-04-06T16:13:07.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:10.626 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-06T16:13:16.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:19.710 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-06T16:13:22.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:22.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:25.698 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-04-06T16:13:25.699 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113
2024-04-06T16:13:25.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:26.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:28.766 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-06T16:13:31.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:31.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:34.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:34.782 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-04-06T16:13:34.782 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113
2024-04-06T16:13:37.826 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-06T16:13:40.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:40.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:43.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:43.842 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-04-06T16:13:43.842 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113
2024-04-06T16:13:45.122 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2024-04-06T16:13:46.910 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-06T16:13:49.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:49.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:51.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:51.262 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2024-04-06T16:13:52.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:52.926 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-04-06T16:13:52.926 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113
2024-04-06T16:13:55.970 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50
2024-04-06T16:13:57.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:58.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:13:58.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:00.322 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2024-04-06T16:14:01.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:01.986 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-04-06T16:14:01.986 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113
2024-04-06T16:14:06.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:07.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:07.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:09.374 WARNING:teuthology.provision.fog.smithi183:[Errno None] Unable to connect to port 22 on 172.21.15.183
2024-04-06T16:14:11.070 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62
2024-04-06T16:14:11.070 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113
2024-04-06T16:14:15.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:17.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:17.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:20.126 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113
2024-04-06T16:14:26.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:14:29.214 WARNING:teuthology.provision.fog.smithi113:[Errno None] Unable to connect to port 22 on 172.21.15.113
2024-04-06T16:14:35.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:15:02.003 WARNING:teuthology.provision.fog.smithi050:timed out
2024-04-06T16:15:08.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:15:08.285 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-06T16:15:08.286 DEBUG:teuthology.orchestra.run.smithi050:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-06T16:15:08.600 INFO:teuthology.provision.fog.smithi050:Node is ready
2024-04-06T16:15:08.600 DEBUG:teuthology.orchestra.run.smithi050:> hostname
2024-04-06T16:15:08.614 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com
2024-04-06T16:15:08.615 DEBUG:teuthology.orchestra.run.smithi050:> grep smithi050.front.sepia.ceph.com /etc/hosts
2024-04-06T16:15:08.670 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050
2024-04-06T16:15:08.670 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050
2024-04-06T16:15:08.788 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hosts
2024-04-06T16:15:08.818 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/172.21.15.50/172.21.15.50/g' /etc/hosts
2024-04-06T16:15:08.888 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hostname
2024-04-06T16:15:08.960 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050
2024-04-06T16:15:09.030 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release
2024-04-06T16:15:09.084 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-06T16:15:09.085 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/"
2024-04-06T16:15:09.086 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-06T16:15:09.086 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-06T16:15:09.086 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-06T16:15:09.086 INFO:teuthology.provision.fog.smithi050:Deploy complete!
2024-04-06T16:15:09.088 DEBUG:teuthology.parallel:result is None
2024-04-06T16:15:15.387 WARNING:teuthology.provision.fog.smithi183:timed out
2024-04-06T16:15:17.077 WARNING:teuthology.provision.fog.smithi062:timed out
2024-04-06T16:15:21.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:15:21.661 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-06T16:15:21.661 DEBUG:teuthology.orchestra.run.smithi183:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-06T16:15:21.985 INFO:teuthology.provision.fog.smithi183:Node is ready
2024-04-06T16:15:21.986 DEBUG:teuthology.orchestra.run.smithi183:> hostname
2024-04-06T16:15:22.000 INFO:teuthology.orchestra.run.smithi183.stdout:smithi183.front.sepia.ceph.com
2024-04-06T16:15:22.000 DEBUG:teuthology.orchestra.run.smithi183:> grep smithi183.front.sepia.ceph.com /etc/hosts
2024-04-06T16:15:22.056 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.15.183 smithi183.front.sepia.ceph.com smithi183
2024-04-06T16:15:22.056 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183
2024-04-06T16:15:22.163 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hosts
2024-04-06T16:15:22.190 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/172.21.15.183/172.21.15.183/g' /etc/hosts
2024-04-06T16:15:22.258 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i -e 's/smithi183.front.sepia.ceph.com/smithi183/g' /etc/hostname
2024-04-06T16:15:22.327 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname smithi183
2024-04-06T16:15:22.395 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release
2024-04-06T16:15:22.450 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream"
2024-04-06T16:15:22.450 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9"
2024-04-06T16:15:22.450 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos"
2024-04-06T16:15:22.450 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora"
2024-04-06T16:15:22.450 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9"
2024-04-06T16:15:22.450 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9"
2024-04-06T16:15:22.451 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-06T16:15:22.451 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31"
2024-04-06T16:15:22.451 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon"
2024-04-06T16:15:22.451 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-06T16:15:22.451 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/"
2024-04-06T16:15:22.451 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-06T16:15:22.451 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-06T16:15:22.451 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-06T16:15:22.451 INFO:teuthology.provision.fog.smithi183:Deploy complete!
2024-04-06T16:15:22.452 DEBUG:teuthology.parallel:result is None
2024-04-06T16:15:23.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:15:23.350 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-06T16:15:23.350 DEBUG:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-06T16:15:23.693 INFO:teuthology.provision.fog.smithi062:Node is ready
2024-04-06T16:15:23.693 DEBUG:teuthology.orchestra.run.smithi062:> hostname
2024-04-06T16:15:23.708 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com
2024-04-06T16:15:23.708 DEBUG:teuthology.orchestra.run.smithi062:> grep smithi062.front.sepia.ceph.com /etc/hosts
2024-04-06T16:15:23.764 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062
2024-04-06T16:15:23.764 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062
2024-04-06T16:15:23.873 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hosts
2024-04-06T16:15:23.905 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/172.21.15.62/172.21.15.62/g' /etc/hosts
2024-04-06T16:15:23.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hostname
2024-04-06T16:15:24.046 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062
2024-04-06T16:15:24.116 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release
2024-04-06T16:15:24.172 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream"
2024-04-06T16:15:24.173 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9"
2024-04-06T16:15:24.173 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos"
2024-04-06T16:15:24.173 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora"
2024-04-06T16:15:24.173 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9"
2024-04-06T16:15:24.173 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9"
2024-04-06T16:15:24.173 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-06T16:15:24.174 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31"
2024-04-06T16:15:24.174 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon"
2024-04-06T16:15:24.174 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-06T16:15:24.174 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/"
2024-04-06T16:15:24.174 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-06T16:15:24.177 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-06T16:15:24.178 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-06T16:15:24.178 INFO:teuthology.provision.fog.smithi062:Deploy complete!
2024-04-06T16:15:24.180 DEBUG:teuthology.parallel:result is None
2024-04-06T16:15:35.229 WARNING:teuthology.provision.fog.smithi113:timed out
2024-04-06T16:15:41.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-06T16:15:41.560 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-06T16:15:41.560 DEBUG:teuthology.orchestra.run.smithi113:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-06T16:15:41.892 INFO:teuthology.provision.fog.smithi113:Node is ready
2024-04-06T16:15:41.892 DEBUG:teuthology.orchestra.run.smithi113:> hostname
2024-04-06T16:15:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:smithi113.front.sepia.ceph.com
2024-04-06T16:15:41.908 DEBUG:teuthology.orchestra.run.smithi113:> grep smithi113.front.sepia.ceph.com /etc/hosts
2024-04-06T16:15:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113 smithi113.front.sepia.ceph.com smithi113
2024-04-06T16:15:41.970 DEBUG:teuthology.orchestra.run.smithi113:> sudo hostname smithi113
2024-04-06T16:15:42.084 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i -e 's/smithi113.front.sepia.ceph.com/smithi113/g' /etc/hosts
2024-04-06T16:15:42.112 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i -e 's/172.21.15.113/172.21.15.113/g' /etc/hosts
2024-04-06T16:15:42.182 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i -e 's/smithi113.front.sepia.ceph.com/smithi113/g' /etc/hostname
2024-04-06T16:15:42.249 DEBUG:teuthology.orchestra.run.smithi113:> sudo hostname smithi113
2024-04-06T16:15:42.316 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream"
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9"
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos"
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora"
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9"
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9"
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31"
2024-04-06T16:15:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon"
2024-04-06T16:15:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-06T16:15:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/"
2024-04-06T16:15:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-06T16:15:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-06T16:15:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-06T16:15:42.373 INFO:teuthology.provision.fog.smithi113:Deploy complete!
2024-04-06T16:15:42.374 DEBUG:teuthology.parallel:result is None
2024-04-06T16:15:43.610 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-04-06T16:15:43.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-06T16:15:43.906 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=212b46be78984d1f990091f80c44d590
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-04-06T16:15:43.908 INFO:teuthology.nuke:Checking targets against current locks
2024-04-06T16:15:43.977 DEBUG:teuthology.nuke:shortname: smithi050
2024-04-06T16:15:43.977 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-06T16:15:44.008 DEBUG:teuthology.nuke:shortname: smithi062
2024-04-06T16:15:44.008 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-06T16:15:44.010 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-06_14:49:40-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7642491', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-06 16:10:17.840538', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFJ4kGjYAMarCY1gAwyUh3Y1vUq7yqC95zOIqpvZ4+fbp62854/9pDng5p5Ra6UkwWtmumClh7N9/lY52EvLEH8='}
2024-04-06T16:15:44.038 DEBUG:teuthology.nuke:shortname: smithi113
2024-04-06T16:15:44.039 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-06T16:15:44.040 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-06_14:49:40-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7642491', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-06 16:10:17.842669', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOiDUHGVc7H05wu58+py0BtWbGXrvq9zyVvWPPflq9AyQLWM0pogPo+4WCjnYqFR3NB3ry2qjCkAP/feeGdqKw4='}
2024-04-06T16:15:44.058 DEBUG:teuthology.nuke:shortname: smithi183
2024-04-06T16:15:44.058 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-06T16:15:44.065 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-06_14:49:40-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7642491', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-06 16:10:17.843527', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAU5TvG3NA8SeKMHvFc7Yo17EQc6cH2OVc3mdm0RFyeGN4RS5mOXeXFcnhVkrQpgTxQDjqnwfllTRGYl3LisvXQ='}
2024-04-06T16:15:44.075 INFO:teuthology.orchestra.console.smithi050:Power off
2024-04-06T16:15:44.076 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-06T16:15:44.091 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-06_14:49:40-rados-wip-yuri3-testing-2024-04-05-0825-distro-default-smithi/7642496', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-06 16:15:40.585604', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCg3FS4ymhcN0Xbs2PqDnbV8Rat9+ElcVXs7FX2ewTG38AZv2sng4s4IfJBCAD9tZ1RW44dCn19nJvJaMjIoaxg='}
2024-04-06T16:15:44.101 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off
2024-04-06T16:15:44.101 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:44.121 INFO:teuthology.orchestra.console.smithi062:Power off
2024-04-06T16:15:44.122 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-06T16:15:44.134 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-06T16:15:44.146 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off
2024-04-06T16:15:44.147 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:44.167 INFO:teuthology.orchestra.console.smithi113:Power off
2024-04-06T16:15:44.168 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-06T16:15:44.184 INFO:teuthology.orchestra.console.smithi183:Power off
2024-04-06T16:15:44.184 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-06T16:15:44.195 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off
2024-04-06T16:15:44.195 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:44.205 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-04-06T16:15:44.213 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off
2024-04-06T16:15:44.213 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:44.224 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on
2024-04-06T16:15:44.241 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off
2024-04-06T16:15:44.342 INFO:teuthology.orchestra.console.smithi183:Power off completed
2024-04-06T16:15:44.492 INFO:teuthology.lock.ops:unlocked: smithi183.front.sepia.ceph.com
2024-04-06T16:15:48.138 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:48.206 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:48.225 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:48.248 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-06T16:15:48.317 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-04-06T16:15:48.335 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on
2024-04-06T16:15:52.251 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:52.318 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:52.336 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:52.361 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on
2024-04-06T16:15:52.428 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on
2024-04-06T16:15:52.446 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on
2024-04-06T16:15:56.366 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:56.429 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:56.448 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-06T16:15:56.477 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off
2024-04-06T16:15:56.539 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off
2024-04-06T16:15:56.557 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off
2024-04-06T16:15:56.579 INFO:teuthology.orchestra.console.smithi050:Power off completed
2024-04-06T16:15:56.641 INFO:teuthology.orchestra.console.smithi062:Power off completed
2024-04-06T16:15:56.658 INFO:teuthology.orchestra.console.smithi113:Power off completed
2024-04-06T16:15:56.740 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com
2024-04-06T16:15:56.791 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com
2024-04-06T16:15:56.807 INFO:teuthology.lock.ops:unlocked: smithi113.front.sepia.ceph.com
2024-04-06T16:15:57.368 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range