2024-02-18T15:01:50.849 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-18T15:01:50.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T15:01:50.999 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feaf3205c10>, signals=[15]) 2024-02-18T15:01:50.999 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging 2024-02-18T15:01:50.999 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server 2024-02-18T15:01:50.999 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-02-18T15:01:51.025 INFO:teuthology.lock.ops:Start node 'smithi082.front.sepia.ceph.com' reimaging 2024-02-18T15:01:51.025 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: reset os type and version on server 2024-02-18T15:01:51.025 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-02-18T15:01:51.151 DEBUG:teuthology.provision.fog.smithi027:Requesting image smithi_centos_9.stream (ID 10) 2024-02-18T15:01:51.154 DEBUG:teuthology.provision.fog.smithi082:Requesting image smithi_centos_9.stream (ID 10) 2024-02-18T15:01:51.195 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of centos 9.stream 2024-02-18T15:01:51.203 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 9.stream 2024-02-18T15:01:51.297 DEBUG:teuthology.provision.fog.smithi082:Canceling deploy task with ID 802897 2024-02-18T15:01:51.537 INFO:teuthology.orchestra.console.smithi027:Power off 2024-02-18T15:01:51.537 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T15:01:51.558 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2024-02-18T15:01:51.558 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:01:51.578 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-02-18T15:01:51.595 INFO:teuthology.orchestra.console.smithi082:Power off 2024-02-18T15:01:51.595 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T15:01:51.616 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-02-18T15:01:51.616 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:01:51.640 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-02-18T15:01:51.679 INFO:teuthology.orchestra.console.smithi027:Power off completed 2024-02-18T15:01:51.741 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-02-18T15:01:51.780 INFO:teuthology.orchestra.console.smithi027:Power on 2024-02-18T15:01:51.780 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-18T15:01:51.801 DEBUG:teuthology.orchestra.console.smithi027:power on output: Chassis Power Control: Up/On 2024-02-18T15:01:51.802 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:01:51.823 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-02-18T15:01:51.842 INFO:teuthology.orchestra.console.smithi082:Power on 2024-02-18T15:01:51.842 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-18T15:01:51.863 DEBUG:teuthology.orchestra.console.smithi082:power on output: Chassis Power Control: Up/On 2024-02-18T15:01:51.864 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:01:51.885 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-02-18T15:01:55.828 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:01:55.885 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:01:55.937 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-02-18T15:01:55.994 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-02-18T15:01:56.095 INFO:teuthology.orchestra.console.smithi082:Power on completed 2024-02-18T15:01:56.195 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish 2024-02-18T15:01:59.940 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:02:00.049 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-02-18T15:02:00.149 INFO:teuthology.orchestra.console.smithi027:Power on completed 2024-02-18T15:02:00.250 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish 2024-02-18T15:04:27.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:01.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:04.316 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:05:10.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:13.372 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:05:19.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:22.428 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:05:27.124 WARNING:teuthology.provision.fog.smithi082:timed out 2024-02-18T15:05:28.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:31.484 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:05:33.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:37.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:40.540 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:05:46.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:49.596 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:05:55.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:05:58.652 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:06:04.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:06:06.638 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-18T15:06:06.639 DEBUG:teuthology.orchestra.run.smithi082:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-18T15:06:07.711 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:06:13.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:06:16.764 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-18T15:06:22.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:06:26.999 INFO:teuthology.provision.fog.smithi082:Node is ready 2024-02-18T15:06:26.999 DEBUG:teuthology.orchestra.run.smithi082:> hostname 2024-02-18T15:06:27.018 INFO:teuthology.orchestra.run.smithi082.stdout:smithi046 2024-02-18T15:06:27.018 DEBUG:teuthology.orchestra.run.smithi082:> grep smithi046 /etc/hosts 2024-02-18T15:06:27.075 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-02-18T15:06:27.075 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname smithi082 2024-02-18T15:06:27.215 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/smithi046/smithi082/g' /etc/hosts 2024-02-18T15:06:27.265 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/172.21.15.46/172.21.15.82/g' /etc/hosts 2024-02-18T15:06:27.294 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/smithi046/smithi082/g' /etc/hostname 2024-02-18T15:06:27.363 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname smithi082 2024-02-18T15:06:27.440 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-02-18T15:06:27.495 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-02-18T15:06:27.496 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T15:06:27.496 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-02-18T15:06:27.496 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T15:06:27.496 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T15:06:27.496 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T15:06:27.497 INFO:teuthology.provision.fog.smithi082:Deploy complete! 2024-02-18T15:06:27.498 DEBUG:teuthology.parallel:result is None 2024-02-18T15:07:22.821 WARNING:teuthology.provision.fog.smithi027:timed out 2024-02-18T15:07:28.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T15:07:29.085 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-18T15:07:29.085 DEBUG:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-18T15:07:29.421 INFO:teuthology.provision.fog.smithi027:Node is ready 2024-02-18T15:07:29.422 DEBUG:teuthology.orchestra.run.smithi027:> hostname 2024-02-18T15:07:29.437 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com 2024-02-18T15:07:29.438 DEBUG:teuthology.orchestra.run.smithi027:> grep smithi027.front.sepia.ceph.com /etc/hosts 2024-02-18T15:07:29.494 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027 2024-02-18T15:07:29.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027 2024-02-18T15:07:29.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hosts 2024-02-18T15:07:29.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/172.21.15.27/172.21.15.27/g' /etc/hosts 2024-02-18T15:07:29.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hostname 2024-02-18T15:07:29.785 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027 2024-02-18T15:07:29.855 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T15:07:29.911 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T15:07:29.912 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T15:07:29.912 INFO:teuthology.provision.fog.smithi027:Deploy complete! 2024-02-18T15:07:29.912 DEBUG:teuthology.parallel:result is None 2024-02-18T15:07:31.045 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 326, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 290, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/misc.py", line 1108, in ssh_keyscan key = _ssh_keyscan(hostname) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/misc.py", line 1145, in _ssh_keyscan return sorted(keys)[0] IndexError: list index out of range 2024-02-18T15:07:31.046 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T15:07:31.338 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=549b87c5d2bc418c92776e99e8dce411 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 326, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 290, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/misc.py", line 1108, in ssh_keyscan key = _ssh_keyscan(hostname) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/misc.py", line 1145, in _ssh_keyscan return sorted(keys)[0] IndexError: list index out of range 2024-02-18T15:07:31.340 INFO:teuthology.nuke:Checking targets against current locks 2024-02-18T15:07:31.405 DEBUG:teuthology.nuke:shortname: smithi027 2024-02-18T15:07:31.406 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T15:07:31.430 DEBUG:teuthology.nuke:shortname: smithi082 2024-02-18T15:07:31.430 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T15:07:31.433 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-18_11:27:24-fs-wip-lusov-qdb-distro-default-smithi/7565812', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 15:01:49.793370', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyxmxs96sdWuvd+3fr8cNGRRkSo2r8+hqHzhF64l6nz3ubEWc5S0XMGbYGK4y57djF6e/VQysDuAb1fa0xMfII='} 2024-02-18T15:07:31.458 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-02-18_11:27:24-fs-wip-lusov-qdb-distro-default-smithi/7565816', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 15:06:52.181333', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKQQZtWrZ3ULu+JvJxjiT2S7VyfsUsaEqT8G/3l7FsC2f7/1at+wWRtYk0Nlkb9Kyyio27N7wKs9eooyMQVj4Aw='} 2024-02-18T15:07:31.491 INFO:teuthology.orchestra.console.smithi027:Power off 2024-02-18T15:07:31.491 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T15:07:31.522 INFO:teuthology.orchestra.console.smithi082:Power off 2024-02-18T15:07:31.523 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T15:07:31.531 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2024-02-18T15:07:31.532 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:07:31.544 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-02-18T15:07:31.545 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:07:31.557 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-02-18T15:07:31.569 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-02-18T15:07:35.560 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:07:35.570 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:07:35.672 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-02-18T15:07:35.679 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-02-18T15:07:39.678 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:07:39.685 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:07:39.788 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-02-18T15:07:39.795 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-02-18T15:07:39.889 INFO:teuthology.orchestra.console.smithi027:Power off completed 2024-02-18T15:07:40.047 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com 2024-02-18T15:07:43.797 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T15:07:43.907 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-02-18T15:07:44.008 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-02-18T15:07:44.163 INFO:teuthology.lock.ops:unlocked: smithi082.front.sepia.ceph.com 2024-02-18T15:07:44.399 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 326, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 290, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/misc.py", line 1108, in ssh_keyscan key = _ssh_keyscan(hostname) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/misc.py", line 1145, in _ssh_keyscan return sorted(keys)[0] IndexError: list index out of range