2024-03-07T10:38:29.679 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-07T10:38:29.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T10:38:29.828 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f58e11cff10>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f58e2230c10>, signals=[15])
2024-03-07T10:38:29.829 INFO:teuthology.lock.ops:Start node 'smithi003.front.sepia.ceph.com' reimaging
2024-03-07T10:38:29.829 INFO:teuthology.lock.ops:Updating [smithi003.front.sepia.ceph.com]: reset os type and version on server
2024-03-07T10:38:29.829 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server
2024-03-07T10:38:29.855 INFO:teuthology.lock.ops:Start node 'smithi155.front.sepia.ceph.com' reimaging
2024-03-07T10:38:29.856 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: reset os type and version on server
2024-03-07T10:38:29.856 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server
2024-03-07T10:38:29.981 DEBUG:teuthology.provision.fog.smithi003:Requesting image smithi_centos_9.stream (ID 10)
2024-03-07T10:38:29.987 DEBUG:teuthology.provision.fog.smithi155:Requesting image smithi_centos_9.stream (ID 10)
2024-03-07T10:38:30.028 INFO:teuthology.provision.fog.smithi003:Scheduling deploy of centos 9.stream
2024-03-07T10:38:30.037 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of centos 9.stream
2024-03-07T10:38:30.153 DEBUG:teuthology.provision.fog.smithi003:Canceling deploy task with ID 837368
2024-03-07T10:38:30.390 INFO:teuthology.orchestra.console.smithi155:Power off
2024-03-07T10:38:30.391 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T10:38:30.417 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off
2024-03-07T10:38:30.417 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:30.439 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on
2024-03-07T10:38:30.458 INFO:teuthology.orchestra.console.smithi003:Power off
2024-03-07T10:38:30.459 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T10:38:30.481 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off
2024-03-07T10:38:30.481 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:30.506 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2024-03-07T10:38:34.444 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:34.507 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:34.552 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on
2024-03-07T10:38:34.617 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2024-03-07T10:38:38.556 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:38.619 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:38.665 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on
2024-03-07T10:38:38.727 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2024-03-07T10:38:42.670 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:42.728 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:42.780 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off
2024-03-07T10:38:42.837 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2024-03-07T10:38:42.881 INFO:teuthology.orchestra.console.smithi155:Power off completed
2024-03-07T10:38:42.937 INFO:teuthology.orchestra.console.smithi003:Power off completed
2024-03-07T10:38:42.982 INFO:teuthology.orchestra.console.smithi155:Power on
2024-03-07T10:38:42.982 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-07T10:38:43.002 DEBUG:teuthology.orchestra.console.smithi155:power on output: Chassis Power Control: Up/On
2024-03-07T10:38:43.003 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:43.023 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off
2024-03-07T10:38:43.039 INFO:teuthology.orchestra.console.smithi003:Power on
2024-03-07T10:38:43.039 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-07T10:38:43.059 DEBUG:teuthology.orchestra.console.smithi003:power on output: Chassis Power Control: Up/On
2024-03-07T10:38:43.059 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:43.079 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2024-03-07T10:38:47.028 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:47.081 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:38:47.138 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on
2024-03-07T10:38:47.190 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on
2024-03-07T10:38:47.239 INFO:teuthology.orchestra.console.smithi155:Power on completed
2024-03-07T10:38:47.291 INFO:teuthology.orchestra.console.smithi003:Power on completed
2024-03-07T10:38:47.340 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish
2024-03-07T10:38:47.391 INFO:teuthology.provision.fog.smithi003:Waiting for deploy to finish
2024-03-07T10:41:34.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:41:37.180 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155
2024-03-07T10:41:43.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:41:46.236 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155
2024-03-07T10:41:49.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:41:52.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:41:55.295 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155
2024-03-07T10:41:59.356 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-03-07T10:42:01.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:04.380 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155
2024-03-07T10:42:05.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:08.412 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-03-07T10:42:10.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:13.440 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155
2024-03-07T10:42:14.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:17.468 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-03-07T10:42:19.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:22.528 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155
2024-03-07T10:42:23.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:26.528 WARNING:teuthology.provision.fog.smithi003:[Errno None] Unable to connect to port 22 on 172.21.15.3
2024-03-07T10:42:28.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:31.612 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155
2024-03-07T10:42:32.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:37.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:42:40.704 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155
2024-03-07T10:42:46.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:43:32.574 WARNING:teuthology.provision.fog.smithi003:timed out
2024-03-07T10:43:38.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:43:38.742 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-07T10:43:38.743 DEBUG:teuthology.orchestra.run.smithi003:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-07T10:43:39.100 INFO:teuthology.provision.fog.smithi003:Node is ready
2024-03-07T10:43:39.100 DEBUG:teuthology.orchestra.run.smithi003:> hostname
2024-03-07T10:43:39.115 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003.front.sepia.ceph.com
2024-03-07T10:43:39.116 DEBUG:teuthology.orchestra.run.smithi003:> grep smithi003.front.sepia.ceph.com /etc/hosts
2024-03-07T10:43:39.171 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3 smithi003.front.sepia.ceph.com smithi003
2024-03-07T10:43:39.172 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003
2024-03-07T10:43:39.272 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hosts
2024-03-07T10:43:39.321 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/172.21.15.3/172.21.15.3/g' /etc/hosts
2024-03-07T10:43:39.352 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i -e 's/smithi003.front.sepia.ceph.com/smithi003/g' /etc/hostname
2024-03-07T10:43:39.437 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname smithi003
2024-03-07T10:43:39.467 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon"
2024-03-07T10:43:39.521 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T10:43:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/"
2024-03-07T10:43:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T10:43:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T10:43:39.522 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T10:43:39.522 INFO:teuthology.provision.fog.smithi003:Deploy complete!
2024-03-07T10:43:39.524 DEBUG:teuthology.parallel:result is None
2024-03-07T10:43:46.721 WARNING:teuthology.provision.fog.smithi155:timed out
2024-03-07T10:43:52.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-07T10:43:53.118 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-07T10:43:53.118 DEBUG:teuthology.orchestra.run.smithi155:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-07T10:43:53.445 INFO:teuthology.provision.fog.smithi155:Node is ready
2024-03-07T10:43:53.445 DEBUG:teuthology.orchestra.run.smithi155:> hostname
2024-03-07T10:43:53.460 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com
2024-03-07T10:43:53.460 DEBUG:teuthology.orchestra.run.smithi155:> grep smithi155.front.sepia.ceph.com /etc/hosts
2024-03-07T10:43:53.515 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155
2024-03-07T10:43:53.515 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname smithi155
2024-03-07T10:43:53.617 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi155.front.sepia.ceph.com/smithi155/g' /etc/hosts
2024-03-07T10:43:53.645 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/172.21.15.155/172.21.15.155/g' /etc/hosts
2024-03-07T10:43:53.712 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi155.front.sepia.ceph.com/smithi155/g' /etc/hostname
2024-03-07T10:43:53.778 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname smithi155
2024-03-07T10:43:53.845 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release
2024-03-07T10:43:53.899 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-07T10:43:53.900 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-07T10:43:53.901 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-07T10:43:53.901 INFO:teuthology.provision.fog.smithi155:Deploy complete!
2024-03-07T10:43:53.901 DEBUG:teuthology.parallel:result is None
2024-03-07T10:43:55.007 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-03-07T10:43:55.009 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-07T10:43:55.284 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=d72f67037aa64d9ba4edc3d202069edc
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-03-07T10:43:55.286 INFO:teuthology.nuke:Checking targets against current locks
2024-03-07T10:43:55.340 DEBUG:teuthology.nuke:shortname: smithi003
2024-03-07T10:43:55.340 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-03-07T10:43:55.362 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7584285', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 10:43:52.717734', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNi1+JRb+rC7tYUc5boSrxhR6PGhe4GvFmw0YACbml6aqAkKPBwUhLENZs46+meafeU1ueSmUoUf+y18acHSCA='}
2024-03-07T10:43:55.365 DEBUG:teuthology.nuke:shortname: smithi155
2024-03-07T10:43:55.365 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-03-07T10:43:55.378 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-07_10:36:38-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7584277', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 10:38:28.629695', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDp+54/I9K4sD2NB5TcvpD84vO0j4cL/WuhwCKOnAFKJTRcGgQy50HKY0y0/Ct3haiZrsm0KLQfDMpX+FLb9Gqo='}
2024-03-07T10:43:55.400 INFO:teuthology.orchestra.console.smithi003:Power off
2024-03-07T10:43:55.401 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T10:43:55.425 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off
2024-03-07T10:43:55.425 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:43:55.438 INFO:teuthology.orchestra.console.smithi155:Power off
2024-03-07T10:43:55.438 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-07T10:43:55.446 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off
2024-03-07T10:43:55.460 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off
2024-03-07T10:43:55.460 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:43:55.482 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on
2024-03-07T10:43:55.547 INFO:teuthology.orchestra.console.smithi003:Power off completed
2024-03-07T10:43:55.695 INFO:teuthology.lock.ops:unlocked: smithi003.front.sepia.ceph.com
2024-03-07T10:43:59.487 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:43:59.595 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on
2024-03-07T10:44:03.598 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:44:03.708 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on
2024-03-07T10:44:07.713 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-07T10:44:07.821 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off
2024-03-07T10:44:07.922 INFO:teuthology.orchestra.console.smithi155:Power off completed
2024-03-07T10:44:08.077 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com
2024-03-07T10:44:08.349 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range