2023-11-29T15:19:44.903 INFO:root:teuthology version: 0.0.1.dev212+g735a43c
2023-11-29T15:19:44.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T15:19:45.040 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fe8088a7dc0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fe809996550>, signals=[15])
2023-11-29T15:19:45.041 INFO:teuthology.lock.ops:Start node 'smithi099.front.sepia.ceph.com' reimaging
2023-11-29T15:19:45.041 INFO:teuthology.lock.ops:Updating [smithi099.front.sepia.ceph.com]: reset os type and version on server
2023-11-29T15:19:45.041 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server
2023-11-29T15:19:45.064 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging
2023-11-29T15:19:45.064 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server
2023-11-29T15:19:45.064 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2023-11-29T15:19:45.194 DEBUG:teuthology.provision.fog.smithi099:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-11-29T15:19:45.199 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_ubuntu_22.04 (ID 6)
2023-11-29T15:19:45.239 INFO:teuthology.provision.fog.smithi099:Scheduling deploy of ubuntu 22.04
2023-11-29T15:19:45.247 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of ubuntu 22.04
2023-11-29T15:19:45.876 INFO:teuthology.orchestra.console.smithi181:Power off
2023-11-29T15:19:45.876 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-29T15:19:45.887 INFO:teuthology.orchestra.console.smithi099:Power off
2023-11-29T15:19:45.887 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-29T15:19:45.895 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off
2023-11-29T15:19:45.895 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:45.906 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off
2023-11-29T15:19:45.907 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:45.914 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2023-11-29T15:19:45.927 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2023-11-29T15:19:49.919 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:49.928 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:50.027 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2023-11-29T15:19:50.036 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2023-11-29T15:19:54.030 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:54.037 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:54.140 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2023-11-29T15:19:54.144 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2023-11-29T15:19:54.245 INFO:teuthology.orchestra.console.smithi099:Power off completed
2023-11-29T15:19:54.345 INFO:teuthology.orchestra.console.smithi099:Power on
2023-11-29T15:19:54.346 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-29T15:19:54.366 DEBUG:teuthology.orchestra.console.smithi099:power on output: Chassis Power Control: Up/On
2023-11-29T15:19:54.367 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:54.387 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2023-11-29T15:19:58.145 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:58.253 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2023-11-29T15:19:58.354 INFO:teuthology.orchestra.console.smithi181:Power off completed
2023-11-29T15:19:58.388 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:58.455 INFO:teuthology.orchestra.console.smithi181:Power on
2023-11-29T15:19:58.455 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-11-29T15:19:58.475 DEBUG:teuthology.orchestra.console.smithi181:power on output: Chassis Power Control: Up/On
2023-11-29T15:19:58.475 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:19:58.495 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2023-11-29T15:19:58.498 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2023-11-29T15:20:02.498 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:20:02.505 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:20:02.607 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2023-11-29T15:20:02.613 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2023-11-29T15:20:02.713 INFO:teuthology.orchestra.console.smithi099:Power on completed
2023-11-29T15:20:02.814 INFO:teuthology.provision.fog.smithi099:Waiting for deploy to finish
2023-11-29T15:20:06.610 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:20:06.719 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2023-11-29T15:20:06.819 INFO:teuthology.orchestra.console.smithi181:Power on completed
2023-11-29T15:20:06.920 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish
2023-11-29T15:20:07.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:20:10.210 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:20:16.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:20:19.301 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:20:25.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:20:28.386 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:20:34.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:20:37.442 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:20:43.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:20:46.530 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:20:52.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:20:55.589 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:21:01.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:21:04.675 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:21:10.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:21:29.186 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:21:35.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:21:38.242 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:21:44.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:21:44.375 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:21:48.502 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:21:55.625 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:22:05.746 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:22:18.873 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:22:34.998 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:22:54.082 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:23:16.166 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:23:41.249 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:24:09.374 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:24:09.374 WARNING:teuthology.provision.fog.smithi181:'connect to smithi181.front.sepia.ceph.com' reached maximum tries (11) after waiting for 145 seconds
2023-11-29T15:24:15.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:24:15.460 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:24:19.582 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:24:26.665 ERROR:teuthology.orchestra.connection:Error authenticating with smithi181.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2023-11-29T15:25:11.138 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:25:17.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:25:20.198 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:25:26.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:25:29.250 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:25:35.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:25:45.442 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:25:51.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:25:54.502 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:26:00.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:03.554 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:26:09.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:12.610 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:26:18.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:18.613 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:26:24.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:24.616 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:26:25.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:28.386 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:26:30.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:30.619 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2023-11-29T15:26:34.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:36.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:36.829 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-29T15:26:36.830 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-29T15:26:37.442 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:26:37.836 INFO:teuthology.provision.fog.smithi181:Node is ready
2023-11-29T15:26:37.837 DEBUG:teuthology.orchestra.run.smithi181:> hostname
2023-11-29T15:26:37.882 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com
2023-11-29T15:26:37.882 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts
2023-11-29T15:26:37.930 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181
2023-11-29T15:26:37.930 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181
2023-11-29T15:26:38.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts
2023-11-29T15:26:38.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts
2023-11-29T15:26:38.125 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname
2023-11-29T15:26:38.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181
2023-11-29T15:26:38.228 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Ubuntu"
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="22.04"
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_CODENAME=jammy
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:ID=ubuntu
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE=debian
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.ubuntu.com/"
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-11-29T15:26:38.274 INFO:teuthology.orchestra.run.smithi181.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-11-29T15:26:38.275 INFO:teuthology.orchestra.run.smithi181.stdout:UBUNTU_CODENAME=jammy
2023-11-29T15:26:38.275 INFO:teuthology.provision.fog.smithi181:Deploy complete!
2023-11-29T15:26:38.276 DEBUG:teuthology.parallel:result is None
2023-11-29T15:26:43.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:46.498 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:26:52.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:26:55.554 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:27:01.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:27:04.610 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:27:10.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:27:13.666 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:27:19.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:27:38.082 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:27:44.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:27:47.138 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:27:53.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:27:53.141 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:27:59.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:27:59.144 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:28:05.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:28:05.147 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:28:11.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:28:11.151 WARNING:teuthology.provision.fog.smithi099:[Errno None] Unable to connect to port 22 on 172.21.15.99
2023-11-29T15:28:17.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-11-29T15:28:17.369 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-11-29T15:28:17.369 DEBUG:teuthology.orchestra.run.smithi099:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-11-29T15:28:17.821 INFO:teuthology.provision.fog.smithi099:Node is ready
2023-11-29T15:28:17.821 DEBUG:teuthology.orchestra.run.smithi099:> hostname
2023-11-29T15:28:17.866 INFO:teuthology.orchestra.run.smithi099.stdout:smithi099.front.sepia.ceph.com
2023-11-29T15:28:17.866 DEBUG:teuthology.orchestra.run.smithi099:> grep smithi099.front.sepia.ceph.com /etc/hosts
2023-11-29T15:28:17.915 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.15.99 smithi099.front.sepia.ceph.com smithi099
2023-11-29T15:28:17.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099
2023-11-29T15:28:18.057 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hosts
2023-11-29T15:28:18.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/172.21.15.99/172.21.15.99/g' /etc/hosts
2023-11-29T15:28:18.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i -e 's/smithi099.front.sepia.ceph.com/smithi099/g' /etc/hostname
2023-11-29T15:28:18.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname smithi099
2023-11-29T15:28:18.237 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Ubuntu"
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="22.04"
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_CODENAME=jammy
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:ID=ubuntu
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE=debian
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.ubuntu.com/"
2023-11-29T15:28:18.286 INFO:teuthology.orchestra.run.smithi099.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2023-11-29T15:28:18.287 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2023-11-29T15:28:18.287 INFO:teuthology.orchestra.run.smithi099.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2023-11-29T15:28:18.287 INFO:teuthology.orchestra.run.smithi099.stdout:UBUNTU_CODENAME=jammy
2023-11-29T15:28:18.287 INFO:teuthology.provision.fog.smithi099:Deploy complete!
2023-11-29T15:28:18.287 DEBUG:teuthology.parallel:result is None
2023-11-29T15:28:19.426 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2023-11-29T15:28:19.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-11-29T15:28:19.762 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=56d1657d5a994c0f8913bf8bf886e9dd
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2023-11-29T15:28:19.764 INFO:teuthology.nuke:Checking targets against current locks
2023-11-29T15:28:19.826 DEBUG:teuthology.nuke:shortname: smithi099
2023-11-29T15:28:19.826 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-29T15:28:19.846 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2023-11-29_15:17:59-rados-aclamk-test-sb-to-b-vselector-2-distro-default-smithi/7470751', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-29 15:19:44.085917', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFU5BSOUAa6QiwfcwTA7Udx5U3xtor2r5zplKJZcmOW2oFONrhzKmQJy7rkR6qgjou2g0TphoHbbhIM3PA3Zak8='}
2023-11-29T15:28:19.851 DEBUG:teuthology.nuke:shortname: smithi181
2023-11-29T15:28:19.851 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-11-29T15:28:19.870 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2023-11-29_15:17:59-rados-aclamk-test-sb-to-b-vselector-2-distro-default-smithi/7470752', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-29 15:19:44.431067', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIjiWL7rp8a3xy+zaDJVMLf3BXvZte5X8RIEzGYb0Isnf4/YVImpsm6Pc0/97ngpEBXmDSoGb/Cok6I/fDXitOo='}
2023-11-29T15:28:19.893 INFO:teuthology.orchestra.console.smithi099:Power off
2023-11-29T15:28:19.893 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-29T15:28:19.916 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off
2023-11-29T15:28:19.916 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:28:19.930 INFO:teuthology.orchestra.console.smithi181:Power off
2023-11-29T15:28:19.930 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-11-29T15:28:19.938 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2023-11-29T15:28:20.020 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off
2023-11-29T15:28:20.020 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:28:20.124 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2023-11-29T15:28:23.939 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:28:24.049 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2023-11-29T15:28:24.125 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:28:24.235 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2023-11-29T15:28:24.336 INFO:teuthology.orchestra.console.smithi181:Power off completed
2023-11-29T15:28:24.494 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com
2023-11-29T15:28:28.050 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:28:28.160 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on
2023-11-29T15:28:32.161 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-11-29T15:28:32.271 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off
2023-11-29T15:28:32.372 INFO:teuthology.orchestra.console.smithi099:Power off completed
2023-11-29T15:28:32.520 INFO:teuthology.lock.ops:unlocked: smithi099.front.sepia.ceph.com
2023-11-29T15:28:32.763 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_735a43c3d8e74286f980f1d14817fb093cd8147e/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range