2024-03-11T09:06:26.374 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-11T09:06:26.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T09:06:26.560 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fc16c9fbf40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fc16da6ec10>, signals=[15])
2024-03-11T09:06:26.561 INFO:teuthology.lock.ops:Start node 'smithi046.front.sepia.ceph.com' reimaging
2024-03-11T09:06:26.561 INFO:teuthology.lock.ops:Updating [smithi046.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T09:06:26.561 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server
2024-03-11T09:06:26.577 INFO:teuthology.lock.ops:Start node 'smithi133.front.sepia.ceph.com' reimaging
2024-03-11T09:06:26.578 INFO:teuthology.lock.ops:Updating [smithi133.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T09:06:26.578 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server
2024-03-11T09:06:26.594 INFO:teuthology.lock.ops:Start node 'smithi181.front.sepia.ceph.com' reimaging
2024-03-11T09:06:26.594 INFO:teuthology.lock.ops:Updating [smithi181.front.sepia.ceph.com]: reset os type and version on server
2024-03-11T09:06:26.594 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server
2024-03-11T09:06:26.685 DEBUG:teuthology.provision.fog.smithi046:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-03-11T09:06:26.697 DEBUG:teuthology.provision.fog.smithi133:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-03-11T09:06:26.710 DEBUG:teuthology.provision.fog.smithi181:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-03-11T09:06:26.728 INFO:teuthology.provision.fog.smithi046:Scheduling deploy of ubuntu 22.04
2024-03-11T09:06:26.737 INFO:teuthology.provision.fog.smithi133:Scheduling deploy of ubuntu 22.04
2024-03-11T09:06:26.759 INFO:teuthology.provision.fog.smithi181:Scheduling deploy of ubuntu 22.04
2024-03-11T09:06:26.916 DEBUG:teuthology.provision.fog.smithi133:Canceling deploy task with ID 848841
2024-03-11T09:06:27.235 INFO:teuthology.orchestra.console.smithi046:Power off
2024-03-11T09:06:27.236 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T09:06:27.249 INFO:teuthology.orchestra.console.smithi181:Power off
2024-03-11T09:06:27.249 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T09:06:27.256 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off
2024-03-11T09:06:27.257 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:27.272 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off
2024-03-11T09:06:27.272 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:27.282 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off
2024-03-11T09:06:27.296 INFO:teuthology.orchestra.console.smithi133:Power off
2024-03-11T09:06:27.297 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T09:06:27.307 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2024-03-11T09:06:27.320 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off
2024-03-11T09:06:27.320 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:27.345 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on
2024-03-11T09:06:27.382 INFO:teuthology.orchestra.console.smithi046:Power off completed
2024-03-11T09:06:27.483 INFO:teuthology.orchestra.console.smithi046:Power on
2024-03-11T09:06:27.483 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T09:06:27.506 DEBUG:teuthology.orchestra.console.smithi046:power on output: Chassis Power Control: Up/On
2024-03-11T09:06:27.509 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:27.533 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off
2024-03-11T09:06:31.310 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:31.346 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:31.420 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2024-03-11T09:06:31.455 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on
2024-03-11T09:06:31.534 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:31.643 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off
2024-03-11T09:06:35.422 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:35.457 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:35.531 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2024-03-11T09:06:35.566 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off
2024-03-11T09:06:35.633 INFO:teuthology.orchestra.console.smithi181:Power off completed
2024-03-11T09:06:35.644 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:35.667 INFO:teuthology.orchestra.console.smithi133:Power off completed
2024-03-11T09:06:35.734 INFO:teuthology.orchestra.console.smithi181:Power on
2024-03-11T09:06:35.735 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T09:06:35.753 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on
2024-03-11T09:06:35.756 DEBUG:teuthology.orchestra.console.smithi181:power on output: Chassis Power Control: Up/On
2024-03-11T09:06:35.756 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:35.768 INFO:teuthology.orchestra.console.smithi133:Power on
2024-03-11T09:06:35.769 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-11T09:06:35.779 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2024-03-11T09:06:35.790 DEBUG:teuthology.orchestra.console.smithi133:power on output: Chassis Power Control: Up/On
2024-03-11T09:06:35.790 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:35.815 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off
2024-03-11T09:06:35.854 INFO:teuthology.orchestra.console.smithi046:Power on completed
2024-03-11T09:06:35.955 INFO:teuthology.provision.fog.smithi046:Waiting for deploy to finish
2024-03-11T09:06:39.786 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:39.815 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:39.897 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2024-03-11T09:06:39.924 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off
2024-03-11T09:06:43.902 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:43.925 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:06:44.013 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2024-03-11T09:06:44.032 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on
2024-03-11T09:06:44.114 INFO:teuthology.orchestra.console.smithi181:Power on completed
2024-03-11T09:06:44.133 INFO:teuthology.orchestra.console.smithi133:Power on completed
2024-03-11T09:06:44.215 INFO:teuthology.provision.fog.smithi181:Waiting for deploy to finish
2024-03-11T09:06:44.234 INFO:teuthology.provision.fog.smithi133:Waiting for deploy to finish
2024-03-11T09:11:49.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:11:52.222 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:11:58.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:12:01.278 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:12:07.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:12:10.334 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:12:16.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:12:19.390 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:12:25.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:12:28.446 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:12:34.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:12:37.502 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:12:43.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:12:46.558 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:12:52.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:10.942 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:13:16.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:19.998 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:13:26.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:29.090 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:13:34.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:35.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:35.093 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:13:41.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:41.102 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:13:41.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:44.958 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2024-03-11T09:13:47.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:47.108 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:13:50.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:53.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:53.113 WARNING:teuthology.provision.fog.smithi181:[Errno None] Unable to connect to port 22 on 172.21.15.181
2024-03-11T09:13:54.014 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2024-03-11T09:13:59.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:13:59.343 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T09:13:59.343 DEBUG:teuthology.orchestra.run.smithi181:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T09:13:59.849 INFO:teuthology.provision.fog.smithi181:Node is ready
2024-03-11T09:13:59.849 DEBUG:teuthology.orchestra.run.smithi181:> hostname
2024-03-11T09:13:59.897 INFO:teuthology.orchestra.run.smithi181.stdout:smithi181.front.sepia.ceph.com
2024-03-11T09:13:59.897 DEBUG:teuthology.orchestra.run.smithi181:> grep smithi181.front.sepia.ceph.com /etc/hosts
2024-03-11T09:13:59.945 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.15.181 smithi181.front.sepia.ceph.com smithi181
2024-03-11T09:13:59.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181
2024-03-11T09:14:00.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:14:00.063 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hosts
2024-03-11T09:14:00.077 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/172.21.15.181/172.21.15.181/g' /etc/hosts
2024-03-11T09:14:00.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i -e 's/smithi181.front.sepia.ceph.com/smithi181/g' /etc/hostname
2024-03-11T09:14:00.188 DEBUG:teuthology.orchestra.run.smithi181:> sudo hostname smithi181
2024-03-11T09:14:00.244 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release
2024-03-11T09:14:00.297 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-03-11T09:14:00.297 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Ubuntu"
2024-03-11T09:14:00.297 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="22.04"
2024-03-11T09:14:00.297 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-03-11T09:14:00.298 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_CODENAME=jammy
2024-03-11T09:14:00.298 INFO:teuthology.orchestra.run.smithi181.stdout:ID=ubuntu
2024-03-11T09:14:00.298 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE=debian
2024-03-11T09:14:00.298 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.ubuntu.com/"
2024-03-11T09:14:00.298 INFO:teuthology.orchestra.run.smithi181.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-03-11T09:14:00.298 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-03-11T09:14:00.299 INFO:teuthology.orchestra.run.smithi181.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-03-11T09:14:00.299 INFO:teuthology.orchestra.run.smithi181.stdout:UBUNTU_CODENAME=jammy
2024-03-11T09:14:00.300 INFO:teuthology.provision.fog.smithi181:Deploy complete!
2024-03-11T09:14:00.302 DEBUG:teuthology.parallel:result is None
2024-03-11T09:14:03.070 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2024-03-11T09:14:09.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:14:12.158 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2024-03-11T09:14:18.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:14:21.214 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2024-03-11T09:14:27.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:14:30.270 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2024-03-11T09:14:34.983 WARNING:teuthology.provision.fog.smithi133:timed out
2024-03-11T09:14:36.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:14:40.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:14:51.295 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-11T09:14:57.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:15:00.350 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-11T09:15:06.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:15:06.357 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-11T09:15:12.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:15:12.365 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-11T09:15:18.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:15:18.373 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-11T09:15:24.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:15:24.376 WARNING:teuthology.provision.fog.smithi133:[Errno None] Unable to connect to port 22 on 172.21.15.133
2024-03-11T09:15:30.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:15:30.611 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T09:15:30.612 DEBUG:teuthology.orchestra.run.smithi133:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T09:15:31.070 INFO:teuthology.provision.fog.smithi133:Node is ready
2024-03-11T09:15:31.070 DEBUG:teuthology.orchestra.run.smithi133:> hostname
2024-03-11T09:15:31.118 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133.front.sepia.ceph.com
2024-03-11T09:15:31.119 DEBUG:teuthology.orchestra.run.smithi133:> grep smithi133.front.sepia.ceph.com /etc/hosts
2024-03-11T09:15:31.165 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133 smithi133.front.sepia.ceph.com smithi133
2024-03-11T09:15:31.165 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname smithi133
2024-03-11T09:15:31.296 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/smithi133.front.sepia.ceph.com/smithi133/g' /etc/hosts
2024-03-11T09:15:31.310 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/172.21.15.133/172.21.15.133/g' /etc/hosts
2024-03-11T09:15:31.365 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i -e 's/smithi133.front.sepia.ceph.com/smithi133/g' /etc/hostname
2024-03-11T09:15:31.419 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname smithi133
2024-03-11T09:15:31.475 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Ubuntu"
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="22.04"
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_CODENAME=jammy
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:ID=ubuntu
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE=debian
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.ubuntu.com/"
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-03-11T09:15:31.520 INFO:teuthology.orchestra.run.smithi133.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-03-11T09:15:31.521 INFO:teuthology.orchestra.run.smithi133.stdout:UBUNTU_CODENAME=jammy
2024-03-11T09:15:31.521 INFO:teuthology.provision.fog.smithi133:Deploy complete!
2024-03-11T09:15:31.522 DEBUG:teuthology.parallel:result is None
2024-03-11T09:15:36.281 WARNING:teuthology.provision.fog.smithi046:timed out
2024-03-11T09:15:42.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:15:49.407 WARNING:teuthology.provision.fog.smithi046:[Errno None] Unable to connect to port 22 on 172.21.15.46
2024-03-11T09:15:55.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-11T09:15:55.634 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-11T09:15:55.634 DEBUG:teuthology.orchestra.run.smithi046:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-11T09:15:56.098 INFO:teuthology.provision.fog.smithi046:Node is ready
2024-03-11T09:15:56.099 DEBUG:teuthology.orchestra.run.smithi046:> hostname
2024-03-11T09:15:56.147 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046.front.sepia.ceph.com
2024-03-11T09:15:56.148 DEBUG:teuthology.orchestra.run.smithi046:> grep smithi046.front.sepia.ceph.com /etc/hosts
2024-03-11T09:15:56.195 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046
2024-03-11T09:15:56.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname smithi046
2024-03-11T09:15:56.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/smithi046.front.sepia.ceph.com/smithi046/g' /etc/hosts
2024-03-11T09:15:56.333 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/172.21.15.46/172.21.15.46/g' /etc/hosts
2024-03-11T09:15:56.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i -e 's/smithi046.front.sepia.ceph.com/smithi046/g' /etc/hostname
2024-03-11T09:15:56.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname smithi046
2024-03-11T09:15:56.498 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release
2024-03-11T09:15:56.546 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu"
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="22.04"
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=jammy
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/"
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-03-11T09:15:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=jammy
2024-03-11T09:15:56.548 INFO:teuthology.provision.fog.smithi046:Deploy complete!
2024-03-11T09:15:56.548 DEBUG:teuthology.parallel:result is None
2024-03-11T09:15:57.718 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-03-11T09:15:57.719 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-11T09:15:58.010 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f889a4de2b4e46f2804f6e08ac39b5fc
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-03-11T09:15:58.012 INFO:teuthology.nuke:Checking targets against current locks
2024-03-11T09:15:58.074 INFO:teuthology.nuke:Not nuking smithi133.front.sepia.ceph.com because description doesn't match: rfriedma-2024-03-11_09:06:05-smoke-wip-rf-compactref-distro-default-smithi != None
2024-03-11T09:15:58.075 DEBUG:teuthology.nuke:shortname: smithi046
2024-03-11T09:15:58.075 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-03-11T09:15:58.098 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2024-03-11_09:06:05-smoke-wip-rf-compactref-distro-default-smithi/7590818', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 09:06:25.171269', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMA48vsNdlxBOjlKZbsIbNTYA2CUZvgX/9fK1ZBFlcq86kr7QqnF3FF7pODT2Bqy2wWbbXBd2L2T5ktXolXML88='}
2024-03-11T09:15:58.122 DEBUG:teuthology.nuke:shortname: smithi181
2024-03-11T09:15:58.123 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-03-11T09:15:58.145 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rfriedma-2024-03-11_09:06:05-smoke-wip-rf-compactref-distro-default-smithi/7590818', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 09:06:25.173009', 'locked_by': 'scheduled_rfriedma@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJWDjEdz4kp3zl8B8QoPS7WrY8ZJFDWr/nOlsXamL6DcEvwqG71sHAEO381epz5VcW4ME/RlBZ7WQ/5Cx8V8Ap0='}
2024-03-11T09:15:58.155 INFO:teuthology.orchestra.console.smithi046:Power off
2024-03-11T09:15:58.155 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T09:15:58.179 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off
2024-03-11T09:15:58.180 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:15:58.202 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on
2024-03-11T09:15:58.220 INFO:teuthology.orchestra.console.smithi181:Power off
2024-03-11T09:15:58.220 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-11T09:15:58.246 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off
2024-03-11T09:15:58.246 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:15:58.271 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2024-03-11T09:16:02.207 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:16:02.273 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:16:02.323 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on
2024-03-11T09:16:02.382 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2024-03-11T09:16:06.329 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:16:06.383 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:16:06.438 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on
2024-03-11T09:16:06.492 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on
2024-03-11T09:16:10.443 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:16:10.493 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-11T09:16:10.554 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off
2024-03-11T09:16:10.603 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off
2024-03-11T09:16:10.655 INFO:teuthology.orchestra.console.smithi046:Power off completed
2024-03-11T09:16:10.704 INFO:teuthology.orchestra.console.smithi181:Power off completed
2024-03-11T09:16:10.815 INFO:teuthology.lock.ops:unlocked: smithi046.front.sepia.ceph.com
2024-03-11T09:16:10.856 INFO:teuthology.lock.ops:unlocked: smithi181.front.sepia.ceph.com
2024-03-11T09:16:10.858 ERROR:teuthology.nuke:Could not nuke the following targets:
targets:
  smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=
2024-03-11T09:16:11.265 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range