2024-04-18T23:15:39.628 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d
2024-04-18T23:15:39.629 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T23:15:39.904 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f4880eb6f70>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f4881f17c10>, signals=[15])
2024-04-18T23:15:39.904 INFO:teuthology.lock.ops:Start node 'smithi079.front.sepia.ceph.com' reimaging
2024-04-18T23:15:39.905 INFO:teuthology.lock.ops:Updating [smithi079.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T23:15:39.905 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server
2024-04-18T23:15:39.944 INFO:teuthology.lock.ops:Start node 'smithi114.front.sepia.ceph.com' reimaging
2024-04-18T23:15:39.944 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T23:15:39.944 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server
2024-04-18T23:15:39.973 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging
2024-04-18T23:15:39.973 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T23:15:39.973 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server
2024-04-18T23:15:40.000 INFO:teuthology.lock.ops:Start node 'smithi163.front.sepia.ceph.com' reimaging
2024-04-18T23:15:40.000 INFO:teuthology.lock.ops:Updating [smithi163.front.sepia.ceph.com]: reset os type and version on server
2024-04-18T23:15:40.000 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server
2024-04-18T23:15:40.076 DEBUG:teuthology.provision.fog.smithi079:Requesting image smithi_centos_9.stream (ID 10)
2024-04-18T23:15:40.087 DEBUG:teuthology.provision.fog.smithi114:Requesting image smithi_centos_9.stream (ID 10)
2024-04-18T23:15:40.119 DEBUG:teuthology.provision.fog.smithi146:Requesting image smithi_centos_9.stream (ID 10)
2024-04-18T23:15:40.122 INFO:teuthology.provision.fog.smithi079:Scheduling deploy of centos 9.stream
2024-04-18T23:15:40.131 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of centos 9.stream
2024-04-18T23:15:40.140 DEBUG:teuthology.provision.fog.smithi163:Requesting image smithi_centos_9.stream (ID 10)
2024-04-18T23:15:40.181 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 9.stream
2024-04-18T23:15:40.183 INFO:teuthology.provision.fog.smithi163:Scheduling deploy of centos 9.stream
2024-04-18T23:15:40.360 INFO:teuthology.orchestra.console.smithi114:Power off
2024-04-18T23:15:40.360 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T23:15:40.387 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off
2024-04-18T23:15:40.387 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:40.412 INFO:teuthology.orchestra.console.smithi079:Power off
2024-04-18T23:15:40.412 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T23:15:40.425 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2024-04-18T23:15:40.433 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off
2024-04-18T23:15:40.433 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:40.460 INFO:teuthology.orchestra.console.smithi163:Power off
2024-04-18T23:15:40.460 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T23:15:40.496 INFO:teuthology.orchestra.console.smithi146:Power off
2024-04-18T23:15:40.496 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T23:15:40.512 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off
2024-04-18T23:15:40.515 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off
2024-04-18T23:15:40.515 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:40.526 INFO:teuthology.orchestra.console.smithi114:Power off completed
2024-04-18T23:15:40.533 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off
2024-04-18T23:15:40.534 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:40.547 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2024-04-18T23:15:40.561 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off
2024-04-18T23:15:40.613 INFO:teuthology.orchestra.console.smithi079:Power off completed
2024-04-18T23:15:40.627 INFO:teuthology.orchestra.console.smithi114:Power on
2024-04-18T23:15:40.627 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T23:15:40.649 DEBUG:teuthology.orchestra.console.smithi114:power on output: Chassis Power Control: Up/On
2024-04-18T23:15:40.649 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:40.656 INFO:teuthology.orchestra.console.smithi163:Power off completed
2024-04-18T23:15:40.663 INFO:teuthology.orchestra.console.smithi146:Power off completed
2024-04-18T23:15:40.674 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2024-04-18T23:15:40.713 INFO:teuthology.orchestra.console.smithi079:Power on
2024-04-18T23:15:40.713 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T23:15:40.744 DEBUG:teuthology.orchestra.console.smithi079:power on output: Chassis Power Control: Up/On
2024-04-18T23:15:40.744 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:40.761 INFO:teuthology.orchestra.console.smithi163:Power on
2024-04-18T23:15:40.761 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T23:15:40.771 INFO:teuthology.orchestra.console.smithi146:Power on
2024-04-18T23:15:40.771 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-18T23:15:40.786 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off
2024-04-18T23:15:40.805 DEBUG:teuthology.orchestra.console.smithi163:power on output: Chassis Power Control: Up/On
2024-04-18T23:15:40.805 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:40.816 DEBUG:teuthology.orchestra.console.smithi146:power on output: Chassis Power Control: Up/On
2024-04-18T23:15:40.816 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:40.838 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2024-04-18T23:15:40.842 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off
2024-04-18T23:15:44.678 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:44.786 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:44.796 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2024-04-18T23:15:44.839 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:44.848 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:44.896 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off
2024-04-18T23:15:44.950 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2024-04-18T23:15:44.957 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off
2024-04-18T23:15:48.800 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:48.897 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:48.917 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on
2024-04-18T23:15:48.951 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:48.960 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:15:49.018 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on
2024-04-18T23:15:49.019 INFO:teuthology.orchestra.console.smithi114:Power on completed
2024-04-18T23:15:49.074 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on
2024-04-18T23:15:49.120 INFO:teuthology.orchestra.console.smithi079:Power on completed
2024-04-18T23:15:49.120 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish
2024-04-18T23:15:49.176 INFO:teuthology.orchestra.console.smithi146:Power on completed
2024-04-18T23:15:49.185 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2024-04-18T23:15:49.221 INFO:teuthology.provision.fog.smithi079:Waiting for deploy to finish
2024-04-18T23:15:49.280 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish
2024-04-18T23:15:49.288 INFO:teuthology.orchestra.console.smithi163:Power on completed
2024-04-18T23:15:49.388 INFO:teuthology.provision.fog.smithi163:Waiting for deploy to finish
2024-04-18T23:18:20.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:20.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:23.230 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:18:23.231 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:18:29.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:29.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:32.286 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:18:32.318 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:18:35.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:35.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:35.549 ERROR:teuthology.orchestra.connection:Error authenticating with smithi163.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-04-18T23:18:38.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:38.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:41.342 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:18:41.374 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:18:45.538 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:18:47.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:47.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:50.402 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:18:50.434 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:18:51.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:54.594 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:18:56.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:56.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:18:59.458 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:18:59.486 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:19:00.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:03.646 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:19:05.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:05.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:08.510 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:19:08.546 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:19:09.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:12.702 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:19:14.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:14.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:17.598 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:19:18.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:21.758 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:19:23.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:24.702 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:19:27.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:30.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:30.814 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:19:33.758 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:19:33.922 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:19:36.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:39.552 WARNING:teuthology.provision.fog.smithi163:timed out
2024-04-18T23:19:39.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:39.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:42.814 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:19:42.974 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:19:45.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:48.606 WARNING:teuthology.provision.fog.smithi163:[Errno None] Unable to connect to port 22 on 172.21.15.163
2024-04-18T23:19:48.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:48.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:51.870 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114
2024-04-18T23:19:52.030 WARNING:teuthology.provision.fog.smithi079:[Errno None] Unable to connect to port 22 on 172.21.15.79
2024-04-18T23:19:54.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:55.166 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:19:57.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:58.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:19:58.127 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T23:19:58.128 DEBUG:teuthology.orchestra.run.smithi114:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T23:19:58.269 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T23:19:58.269 DEBUG:teuthology.orchestra.run.smithi079:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T23:20:01.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:20:04.222 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:20:10.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:20:13.278 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146
2024-04-18T23:20:13.868 INFO:teuthology.provision.fog.smithi114:Node is ready
2024-04-18T23:20:13.869 DEBUG:teuthology.orchestra.run.smithi114:> hostname
2024-04-18T23:20:13.886 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114.front.sepia.ceph.com
2024-04-18T23:20:13.886 DEBUG:teuthology.orchestra.run.smithi114:> grep smithi114.front.sepia.ceph.com /etc/hosts
2024-04-18T23:20:13.944 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114 smithi114.front.sepia.ceph.com smithi114
2024-04-18T23:20:13.944 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114
2024-04-18T23:20:14.068 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi114.front.sepia.ceph.com/smithi114/g' /etc/hosts
2024-04-18T23:20:14.100 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/172.21.15.114/172.21.15.114/g' /etc/hosts
2024-04-18T23:20:14.171 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi114.front.sepia.ceph.com/smithi114/g' /etc/hostname
2024-04-18T23:20:14.240 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114
2024-04-18T23:20:14.309 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release
2024-04-18T23:20:14.365 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T23:20:14.366 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T23:20:14.367 INFO:teuthology.provision.fog.smithi114:Deploy complete!
2024-04-18T23:20:14.369 DEBUG:teuthology.parallel:result is None
2024-04-18T23:20:19.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:20:19.576 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T23:20:19.576 DEBUG:teuthology.orchestra.run.smithi146:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T23:20:24.229 INFO:teuthology.provision.fog.smithi079:Node is ready
2024-04-18T23:20:24.229 DEBUG:teuthology.orchestra.run.smithi079:> hostname
2024-04-18T23:20:24.247 INFO:teuthology.orchestra.run.smithi079.stdout:smithi079.front.sepia.ceph.com
2024-04-18T23:20:24.248 DEBUG:teuthology.orchestra.run.smithi079:> grep smithi079.front.sepia.ceph.com /etc/hosts
2024-04-18T23:20:24.305 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79 smithi079.front.sepia.ceph.com smithi079
2024-04-18T23:20:24.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079
2024-04-18T23:20:24.442 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hosts
2024-04-18T23:20:24.473 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/172.21.15.79/172.21.15.79/g' /etc/hosts
2024-04-18T23:20:24.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i -e 's/smithi079.front.sepia.ceph.com/smithi079/g' /etc/hostname
2024-04-18T23:20:24.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname smithi079
2024-04-18T23:20:24.680 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release
2024-04-18T23:20:24.735 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream"
2024-04-18T23:20:24.735 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9"
2024-04-18T23:20:24.735 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T23:20:24.736 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T23:20:24.736 INFO:teuthology.provision.fog.smithi079:Deploy complete!
2024-04-18T23:20:24.737 DEBUG:teuthology.parallel:result is None
2024-04-18T23:20:39.872 INFO:teuthology.provision.fog.smithi146:Node is ready
2024-04-18T23:20:39.872 DEBUG:teuthology.orchestra.run.smithi146:> hostname
2024-04-18T23:20:39.891 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com
2024-04-18T23:20:39.891 DEBUG:teuthology.orchestra.run.smithi146:> grep smithi146.front.sepia.ceph.com /etc/hosts
2024-04-18T23:20:39.949 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146
2024-04-18T23:20:39.949 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146
2024-04-18T23:20:40.150 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hosts
2024-04-18T23:20:40.182 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/172.21.15.146/172.21.15.146/g' /etc/hosts
2024-04-18T23:20:40.252 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hostname
2024-04-18T23:20:40.356 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146
2024-04-18T23:20:40.384 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release
2024-04-18T23:20:40.439 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T23:20:40.440 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T23:20:40.441 INFO:teuthology.provision.fog.smithi146:Deploy complete!
2024-04-18T23:20:40.441 DEBUG:teuthology.parallel:result is None
2024-04-18T23:20:54.610 WARNING:teuthology.provision.fog.smithi163:timed out
2024-04-18T23:21:00.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-18T23:21:00.827 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-18T23:21:00.827 DEBUG:teuthology.orchestra.run.smithi163:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-18T23:21:01.163 INFO:teuthology.provision.fog.smithi163:Node is ready
2024-04-18T23:21:01.164 DEBUG:teuthology.orchestra.run.smithi163:> hostname
2024-04-18T23:21:01.178 INFO:teuthology.orchestra.run.smithi163.stdout:smithi163.front.sepia.ceph.com
2024-04-18T23:21:01.178 DEBUG:teuthology.orchestra.run.smithi163:> grep smithi163.front.sepia.ceph.com /etc/hosts
2024-04-18T23:21:01.233 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.15.163 smithi163.front.sepia.ceph.com smithi163
2024-04-18T23:21:01.234 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname smithi163
2024-04-18T23:21:01.338 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163.front.sepia.ceph.com/smithi163/g' /etc/hosts
2024-04-18T23:21:01.366 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/172.21.15.163/172.21.15.163/g' /etc/hosts
2024-04-18T23:21:01.434 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i -e 's/smithi163.front.sepia.ceph.com/smithi163/g' /etc/hostname
2024-04-18T23:21:01.501 DEBUG:teuthology.orchestra.run.smithi163:> sudo hostname smithi163
2024-04-18T23:21:01.568 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release
2024-04-18T23:21:01.622 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="CentOS Stream"
2024-04-18T23:21:01.622 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="9"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:ID="centos"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="rhel fedora"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="9"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el9"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="CentOS Stream 9"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:LOGO="fedora-logo-icon"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://centos.org/"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-04-18T23:21:01.623 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-04-18T23:21:01.623 INFO:teuthology.provision.fog.smithi163:Deploy complete!
2024-04-18T23:21:01.624 DEBUG:teuthology.parallel:result is None
2024-04-18T23:21:02.803 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-04-18T23:21:02.805 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-18T23:21:03.091 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=e4021208d85f4067ad77da979eda9cf0
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-04-18T23:21:03.093 INFO:teuthology.nuke:Checking targets against current locks
2024-04-18T23:21:03.146 DEBUG:teuthology.nuke:shortname: smithi079
2024-04-18T23:21:03.146 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-18T23:21:03.173 DEBUG:teuthology.nuke:shortname: smithi114
2024-04-18T23:21:03.173 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-18T23:21:03.174 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-18_22:49:05-rados-wip-yuri5-testing-2024-04-17-1400-distro-default-smithi/7663033', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 23:15:38.350150', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHr4MD3HlW+m9dzJaSSyMDUOTwEr0DxtKQ4H/qJiNrwHPaDKxCF5IZW7KJ+VToGV+jk0XN4XCTclA8yh24vUM+E='}
2024-04-18T23:21:03.179 INFO:teuthology.nuke:Not nuking smithi146.front.sepia.ceph.com because description doesn't match: yuriw-2024-04-18_22:49:05-rados-wip-yuri5-testing-2024-04-17-1400-distro-default-smithi != None
2024-04-18T23:21:03.198 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-18_22:49:05-rados-wip-yuri5-testing-2024-04-17-1400-distro-default-smithi/7663033', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 23:15:38.351894', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLPK5R3RL9cxTzCeuG9IwiarxyBNYtx3jFFshKGfdWLn9OTMVzTyL9HyfU+7VGEXCh75YEM+5rtgCuTuZvMP+Ps='}
2024-04-18T23:21:03.221 DEBUG:teuthology.nuke:shortname: smithi163
2024-04-18T23:21:03.222 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-18T23:21:03.230 INFO:teuthology.orchestra.console.smithi079:Power off
2024-04-18T23:21:03.231 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T23:21:03.245 INFO:teuthology.orchestra.console.smithi114:Power off
2024-04-18T23:21:03.245 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T23:21:03.253 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off
2024-04-18T23:21:03.254 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:03.264 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-18_22:49:05-rados-wip-yuri5-testing-2024-04-17-1400-distro-default-smithi/7663033', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 23:15:38.348412', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKQ94I4Fcm9N8ClFGQP8osahfVfKc0WfBGsclmDZv5WSMAywzADKNsqxxlIQmTD38MewvsDZlM4rSP1WdAF3V6M='}
2024-04-18T23:21:03.268 DEBUG:teuthology.orchestra.console.smithi114:power off output: Chassis Power Control: Down/Off
2024-04-18T23:21:03.269 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:03.277 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on
2024-04-18T23:21:03.291 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on
2024-04-18T23:21:03.312 INFO:teuthology.orchestra.console.smithi163:Power off
2024-04-18T23:21:03.313 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-18T23:21:03.334 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off
2024-04-18T23:21:03.335 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:03.357 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2024-04-18T23:21:07.278 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:07.292 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:07.358 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:07.388 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on
2024-04-18T23:21:07.402 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is on
2024-04-18T23:21:07.469 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on
2024-04-18T23:21:11.389 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:11.403 DEBUG:teuthology.orchestra.console.smithi114:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:11.470 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-18T23:21:11.498 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off
2024-04-18T23:21:11.513 DEBUG:teuthology.orchestra.console.smithi114:check power output: Chassis Power is off
2024-04-18T23:21:11.579 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off
2024-04-18T23:21:11.601 INFO:teuthology.orchestra.console.smithi079:Power off completed
2024-04-18T23:21:11.613 INFO:teuthology.orchestra.console.smithi114:Power off completed
2024-04-18T23:21:11.680 INFO:teuthology.orchestra.console.smithi163:Power off completed
2024-04-18T23:21:11.749 INFO:teuthology.lock.ops:unlocked: smithi079.front.sepia.ceph.com
2024-04-18T23:21:11.757 INFO:teuthology.lock.ops:unlocked: smithi114.front.sepia.ceph.com
2024-04-18T23:21:11.824 INFO:teuthology.lock.ops:unlocked: smithi163.front.sepia.ceph.com
2024-04-18T23:21:11.825 ERROR:teuthology.nuke:Could not nuke the following targets:
targets:
  smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPt8nH5HQitG0yvqZkcpLuS/hrYVe0CL3MrYfzD6xrJ55HOnE3uttkWxasEwtse8OUiVd0IKPNWnALBhq/B9HCc=
2024-04-18T23:21:12.340 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range