2024-04-10T19:32:03.876 INFO:root:teuthology version: 0.0.1.dev267+g6c63784
2024-04-10T19:32:03.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-10T19:32:04.379 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fc5d83b3f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fc5d9414c10>, signals=[15])
2024-04-10T19:32:04.379 INFO:teuthology.lock.ops:Start node 'smithi047.front.sepia.ceph.com' reimaging
2024-04-10T19:32:04.379 INFO:teuthology.lock.ops:Updating [smithi047.front.sepia.ceph.com]: reset os type and version on server
2024-04-10T19:32:04.380 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server
2024-04-10T19:32:04.427 INFO:teuthology.lock.ops:Start node 'smithi067.front.sepia.ceph.com' reimaging
2024-04-10T19:32:04.427 INFO:teuthology.lock.ops:Updating [smithi067.front.sepia.ceph.com]: reset os type and version on server
2024-04-10T19:32:04.427 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server
2024-04-10T19:32:04.477 INFO:teuthology.lock.ops:Start node 'smithi135.front.sepia.ceph.com' reimaging
2024-04-10T19:32:04.477 INFO:teuthology.lock.ops:Updating [smithi135.front.sepia.ceph.com]: reset os type and version on server
2024-04-10T19:32:04.477 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server
2024-04-10T19:32:04.534 INFO:teuthology.lock.ops:Start node 'smithi170.front.sepia.ceph.com' reimaging
2024-04-10T19:32:04.534 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: reset os type and version on server
2024-04-10T19:32:04.535 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server
2024-04-10T19:32:04.658 DEBUG:teuthology.provision.fog.smithi047:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-04-10T19:32:04.734 DEBUG:teuthology.provision.fog.smithi067:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-04-10T19:32:04.762 DEBUG:teuthology.provision.fog.smithi135:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-04-10T19:32:04.782 DEBUG:teuthology.provision.fog.smithi170:Requesting image smithi_ubuntu_22.04 (ID 6)
2024-04-10T19:32:04.783 INFO:teuthology.provision.fog.smithi047:Scheduling deploy of ubuntu 22.04
2024-04-10T19:32:04.815 INFO:teuthology.provision.fog.smithi067:Scheduling deploy of ubuntu 22.04
2024-04-10T19:32:04.863 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of ubuntu 22.04
2024-04-10T19:32:04.880 INFO:teuthology.provision.fog.smithi135:Scheduling deploy of ubuntu 22.04
2024-04-10T19:32:05.271 INFO:teuthology.orchestra.console.smithi047:Power off
2024-04-10T19:32:05.271 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-10T19:32:05.282 INFO:teuthology.orchestra.console.smithi067:Power off
2024-04-10T19:32:05.283 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-10T19:32:05.294 INFO:teuthology.orchestra.console.smithi135:Power off
2024-04-10T19:32:05.295 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-10T19:32:05.309 DEBUG:teuthology.orchestra.console.smithi047:power off output: Chassis Power Control: Down/Off
2024-04-10T19:32:05.310 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:05.318 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off
2024-04-10T19:32:05.318 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:05.332 INFO:teuthology.orchestra.console.smithi170:Power off
2024-04-10T19:32:05.332 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-10T19:32:05.340 DEBUG:teuthology.orchestra.console.smithi135:power off output: Chassis Power Control: Down/Off
2024-04-10T19:32:05.341 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:05.348 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off
2024-04-10T19:32:05.353 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off
2024-04-10T19:32:05.371 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off
2024-04-10T19:32:05.371 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:05.397 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off
2024-04-10T19:32:05.408 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off
2024-04-10T19:32:05.449 INFO:teuthology.orchestra.console.smithi047:Power off completed
2024-04-10T19:32:05.453 INFO:teuthology.orchestra.console.smithi067:Power off completed
2024-04-10T19:32:05.498 INFO:teuthology.orchestra.console.smithi135:Power off completed
2024-04-10T19:32:05.510 INFO:teuthology.orchestra.console.smithi170:Power off completed
2024-04-10T19:32:05.550 INFO:teuthology.orchestra.console.smithi047:Power on
2024-04-10T19:32:05.550 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-10T19:32:05.558 INFO:teuthology.orchestra.console.smithi067:Power on
2024-04-10T19:32:05.558 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-10T19:32:05.594 DEBUG:teuthology.orchestra.console.smithi047:power on output: Chassis Power Control: Up/On
2024-04-10T19:32:05.594 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:05.601 INFO:teuthology.orchestra.console.smithi135:Power on
2024-04-10T19:32:05.601 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-10T19:32:05.607 DEBUG:teuthology.orchestra.console.smithi067:power on output: Chassis Power Control: Up/On
2024-04-10T19:32:05.608 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:05.615 INFO:teuthology.orchestra.console.smithi170:Power on
2024-04-10T19:32:05.615 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-04-10T19:32:05.627 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off
2024-04-10T19:32:05.633 DEBUG:teuthology.orchestra.console.smithi135:power on output: Chassis Power Control: Up/On
2024-04-10T19:32:05.633 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:05.641 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off
2024-04-10T19:32:05.648 DEBUG:teuthology.orchestra.console.smithi170:power on output: Chassis Power Control: Up/On
2024-04-10T19:32:05.648 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:05.665 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off
2024-04-10T19:32:05.680 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off
2024-04-10T19:32:09.632 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:09.642 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:09.666 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:09.681 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:09.741 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on
2024-04-10T19:32:09.750 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off
2024-04-10T19:32:09.775 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on
2024-04-10T19:32:09.789 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off
2024-04-10T19:32:09.842 INFO:teuthology.orchestra.console.smithi047:Power on completed
2024-04-10T19:32:09.876 INFO:teuthology.orchestra.console.smithi135:Power on completed
2024-04-10T19:32:09.943 INFO:teuthology.provision.fog.smithi047:Waiting for deploy to finish
2024-04-10T19:32:09.977 INFO:teuthology.provision.fog.smithi135:Waiting for deploy to finish
2024-04-10T19:32:13.755 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:13.790 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:32:13.864 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on
2024-04-10T19:32:13.898 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on
2024-04-10T19:32:13.965 INFO:teuthology.orchestra.console.smithi067:Power on completed
2024-04-10T19:32:13.999 INFO:teuthology.orchestra.console.smithi170:Power on completed
2024-04-10T19:32:14.066 INFO:teuthology.provision.fog.smithi067:Waiting for deploy to finish
2024-04-10T19:32:14.100 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish
2024-04-10T19:35:11.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:26.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:29.822 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47
2024-04-10T19:35:30.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:30.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:33.406 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:35:35.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:38.910 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47
2024-04-10T19:35:39.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:42.494 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:35:44.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:47.999 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47
2024-04-10T19:35:48.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:48.766 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67
2024-04-10T19:35:51.618 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:35:54.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:54.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:57.086 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47
2024-04-10T19:35:57.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:35:57.854 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67
2024-04-10T19:36:00.702 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:36:03.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:03.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:06.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:06.942 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67
2024-04-10T19:36:09.790 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:36:12.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:15.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:16.030 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67
2024-04-10T19:36:22.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:25.118 WARNING:teuthology.provision.fog.smithi067:[Errno None] Unable to connect to port 22 on 172.21.15.67
2024-04-10T19:36:26.372 WARNING:teuthology.provision.fog.smithi135:timed out
2024-04-10T19:36:31.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:32.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:34.334 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:36:38.495 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2024-04-10T19:36:40.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:43.422 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:36:44.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:45.598 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2024-04-10T19:36:49.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:49.621 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:36:51.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:54.687 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2024-04-10T19:36:55.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:36:55.636 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:37:00.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:01.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:01.651 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:37:02.751 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2024-04-10T19:37:03.135 WARNING:teuthology.provision.fog.smithi047:timed out
2024-04-10T19:37:07.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:07.671 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:37:08.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:08.769 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2024-04-10T19:37:09.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:13.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:13.691 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170
2024-04-10T19:37:14.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:14.784 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2024-04-10T19:37:16.319 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47
2024-04-10T19:37:19.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:19.964 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-10T19:37:19.965 DEBUG:teuthology.orchestra.run.smithi170:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-10T19:37:20.397 INFO:teuthology.provision.fog.smithi170:Node is ready
2024-04-10T19:37:20.398 DEBUG:teuthology.orchestra.run.smithi170:> hostname
2024-04-10T19:37:20.446 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com
2024-04-10T19:37:20.446 DEBUG:teuthology.orchestra.run.smithi170:> grep smithi170.front.sepia.ceph.com /etc/hosts
2024-04-10T19:37:20.498 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170
2024-04-10T19:37:20.499 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname smithi170
2024-04-10T19:37:20.674 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hosts
2024-04-10T19:37:20.689 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/172.21.15.170/172.21.15.170/g' /etc/hosts
2024-04-10T19:37:20.741 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hostname
2024-04-10T19:37:20.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:20.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname smithi170
2024-04-10T19:37:20.800 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2024-04-10T19:37:20.854 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Ubuntu"
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="22.04"
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_CODENAME=jammy
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:ID=ubuntu
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE=debian
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.ubuntu.com/"
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-04-10T19:37:20.903 INFO:teuthology.orchestra.run.smithi170.stdout:UBUNTU_CODENAME=jammy
2024-04-10T19:37:20.904 INFO:teuthology.provision.fog.smithi170:Deploy complete!
2024-04-10T19:37:20.906 DEBUG:teuthology.parallel:result is None
2024-04-10T19:37:22.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:22.509 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-10T19:37:22.509 DEBUG:teuthology.orchestra.run.smithi047:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-10T19:37:22.972 INFO:teuthology.provision.fog.smithi047:Node is ready
2024-04-10T19:37:22.973 DEBUG:teuthology.orchestra.run.smithi047:> hostname
2024-04-10T19:37:23.021 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047.front.sepia.ceph.com
2024-04-10T19:37:23.021 DEBUG:teuthology.orchestra.run.smithi047:> grep smithi047.front.sepia.ceph.com /etc/hosts
2024-04-10T19:37:23.070 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47 smithi047.front.sepia.ceph.com smithi047
2024-04-10T19:37:23.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname smithi047
2024-04-10T19:37:23.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/smithi047.front.sepia.ceph.com/smithi047/g' /etc/hosts
2024-04-10T19:37:23.248 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/172.21.15.47/172.21.15.47/g' /etc/hosts
2024-04-10T19:37:23.301 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/smithi047.front.sepia.ceph.com/smithi047/g' /etc/hostname
2024-04-10T19:37:23.357 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname smithi047
2024-04-10T19:37:23.413 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release
2024-04-10T19:37:23.461 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-04-10T19:37:23.461 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="Ubuntu"
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="22.04"
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_CODENAME=jammy
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:ID=ubuntu
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE=debian
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.ubuntu.com/"
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-04-10T19:37:23.462 INFO:teuthology.orchestra.run.smithi047.stdout:UBUNTU_CODENAME=jammy
2024-04-10T19:37:23.463 INFO:teuthology.provision.fog.smithi047:Deploy complete!
2024-04-10T19:37:23.463 DEBUG:teuthology.parallel:result is None
2024-04-10T19:37:26.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:26.813 WARNING:teuthology.provision.fog.smithi135:[Errno None] Unable to connect to port 22 on 172.21.15.135
2024-04-10T19:37:31.148 WARNING:teuthology.provision.fog.smithi067:timed out
2024-04-10T19:37:32.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:33.023 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-10T19:37:33.023 DEBUG:teuthology.orchestra.run.smithi135:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-10T19:37:33.635 INFO:teuthology.provision.fog.smithi135:Node is ready
2024-04-10T19:37:33.636 DEBUG:teuthology.orchestra.run.smithi135:> hostname
2024-04-10T19:37:33.683 INFO:teuthology.orchestra.run.smithi135.stdout:smithi135.front.sepia.ceph.com
2024-04-10T19:37:33.683 DEBUG:teuthology.orchestra.run.smithi135:> grep smithi135.front.sepia.ceph.com /etc/hosts
2024-04-10T19:37:33.731 INFO:teuthology.orchestra.run.smithi135.stdout:172.21.15.135 smithi135.front.sepia.ceph.com smithi135
2024-04-10T19:37:33.731 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname smithi135
2024-04-10T19:37:33.872 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i -e 's/smithi135.front.sepia.ceph.com/smithi135/g' /etc/hosts
2024-04-10T19:37:33.885 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i -e 's/172.21.15.135/172.21.15.135/g' /etc/hosts
2024-04-10T19:37:33.939 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i -e 's/smithi135.front.sepia.ceph.com/smithi135/g' /etc/hostname
2024-04-10T19:37:33.995 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname smithi135
2024-04-10T19:37:34.053 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="Ubuntu"
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="22.04"
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_CODENAME=jammy
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:ID=ubuntu
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE=debian
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://www.ubuntu.com/"
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-04-10T19:37:34.103 INFO:teuthology.orchestra.run.smithi135.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-04-10T19:37:34.104 INFO:teuthology.orchestra.run.smithi135.stdout:UBUNTU_CODENAME=jammy
2024-04-10T19:37:34.104 INFO:teuthology.provision.fog.smithi135:Deploy complete!
2024-04-10T19:37:34.104 DEBUG:teuthology.parallel:result is None
2024-04-10T19:37:37.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-04-10T19:37:37.388 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-04-10T19:37:37.388 DEBUG:teuthology.orchestra.run.smithi067:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-04-10T19:37:37.812 INFO:teuthology.provision.fog.smithi067:Node is ready
2024-04-10T19:37:37.812 DEBUG:teuthology.orchestra.run.smithi067:> hostname
2024-04-10T19:37:37.862 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067.front.sepia.ceph.com
2024-04-10T19:37:37.862 DEBUG:teuthology.orchestra.run.smithi067:> grep smithi067.front.sepia.ceph.com /etc/hosts
2024-04-10T19:37:37.915 INFO:teuthology.orchestra.run.smithi067.stdout:172.21.15.67 smithi067.front.sepia.ceph.com smithi067
2024-04-10T19:37:37.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname smithi067
2024-04-10T19:37:38.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i -e 's/smithi067.front.sepia.ceph.com/smithi067/g' /etc/hosts
2024-04-10T19:37:38.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i -e 's/172.21.15.67/172.21.15.67/g' /etc/hosts
2024-04-10T19:37:38.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i -e 's/smithi067.front.sepia.ceph.com/smithi067/g' /etc/hostname
2024-04-10T19:37:38.207 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname smithi067
2024-04-10T19:37:38.265 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS"
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="Ubuntu"
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="22.04"
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)"
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_CODENAME=jammy
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:ID=ubuntu
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE=debian
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://www.ubuntu.com/"
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:SUPPORT_URL="https://help.ubuntu.com/"
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2024-04-10T19:37:38.315 INFO:teuthology.orchestra.run.smithi067.stdout:UBUNTU_CODENAME=jammy
2024-04-10T19:37:38.316 INFO:teuthology.provision.fog.smithi067:Deploy complete!
2024-04-10T19:37:38.316 DEBUG:teuthology.parallel:result is None
2024-04-10T19:37:38.706 ERROR:teuthology.misc:write (smithi135.front.sepia.ceph.com): Connection refused
2024-04-10T19:37:38.706 ERROR:teuthology.misc:write (smithi135.front.sepia.ceph.com): Connection refused
2024-04-10T19:37:38.706 ERROR:teuthology.misc:write (smithi135.front.sepia.ceph.com): Connection refused
2024-04-10T19:37:38.706 ERROR:teuthology.misc:write (smithi135.front.sepia.ceph.com): Connection refused
2024-04-10T19:37:38.706 ERROR:teuthology.misc:write (smithi135.front.sepia.ceph.com): Connection refused
2024-04-10T19:37:38.706 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-04-10T19:37:38.708 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-10T19:37:38.984 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b6a2b591d8a34faf836f1fe02dcdc378
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range
2024-04-10T19:37:38.986 INFO:teuthology.nuke:Checking targets against current locks
2024-04-10T19:37:39.124 DEBUG:teuthology.nuke:shortname: smithi047
2024-04-10T19:37:39.124 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-10T19:37:39.177 DEBUG:teuthology.nuke:shortname: smithi067
2024-04-10T19:37:39.177 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-10T19:37:39.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-10_14:17:51-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7651078', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-10 19:32:02.500151', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLUCDA/dhF1OEvuhpg01F6WbtN5+46lvo3fKc7YIflYX74Mlr1C1Hga9LvBCQLLYCgOhyEyKbuVO8GR4lJAPdm0='}
2024-04-10T19:37:39.229 DEBUG:teuthology.nuke:shortname: smithi135
2024-04-10T19:37:39.229 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-10T19:37:39.230 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-10_14:17:51-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7651078', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-10 19:32:02.494897', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5xlyFIOrWMuG5cs2DyTDRXAiK+PUSfqwyFDcJj5p7vvDgC4MGe+8BYJbfKj9MBT5qjgOmiBCQs7e+zvEXKU8A='}
2024-04-10T19:37:39.257 DEBUG:teuthology.nuke:shortname: smithi170
2024-04-10T19:37:39.258 INFO:teuthology.task.internal.check_lock:Checking locks...
2024-04-10T19:37:39.286 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-10_14:17:51-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7651078', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-10 19:32:02.496886', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOdPVlDQeOIR00E/agQjQto0JjvP9DshO76ysDVE3M/n'}
2024-04-10T19:37:39.304 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-10_14:17:51-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7651078', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-10 19:32:02.498617', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgd39/zZqpTPmhoCkna1ojwjG6O0yWmydtzyKJwmjgoXYwH+mvy/yholJ+aFjSj2eju7jFxuZ3c2pPX2uz+akE='}
2024-04-10T19:37:39.328 INFO:teuthology.orchestra.console.smithi047:Power off
2024-04-10T19:37:39.329 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-10T19:37:39.381 DEBUG:teuthology.orchestra.console.smithi047:power off output: Chassis Power Control: Down/Off
2024-04-10T19:37:39.381 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:39.394 INFO:teuthology.orchestra.console.smithi135:Power off
2024-04-10T19:37:39.395 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-10T19:37:39.410 INFO:teuthology.orchestra.console.smithi067:Power off
2024-04-10T19:37:39.410 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-10T19:37:39.421 INFO:teuthology.orchestra.console.smithi170:Power off
2024-04-10T19:37:39.422 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-04-10T19:37:39.430 DEBUG:teuthology.orchestra.console.smithi135:power off output: Chassis Power Control: Down/Off
2024-04-10T19:37:39.430 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:39.437 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on
2024-04-10T19:37:39.444 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off
2024-04-10T19:37:39.444 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:39.458 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off
2024-04-10T19:37:39.458 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:39.465 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is on
2024-04-10T19:37:39.477 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on
2024-04-10T19:37:39.490 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on
2024-04-10T19:37:43.443 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:43.466 DEBUG:teuthology.orchestra.console.smithi135:pexpect command: ipmitool -H smithi135.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:43.478 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:43.491 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:43.553 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on
2024-04-10T19:37:43.575 DEBUG:teuthology.orchestra.console.smithi135:check power output: Chassis Power is off
2024-04-10T19:37:43.586 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on
2024-04-10T19:37:43.600 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on
2024-04-10T19:37:43.676 INFO:teuthology.orchestra.console.smithi135:Power off completed
2024-04-10T19:37:43.842 ERROR:teuthology.lock.ops:failed to unlock smithi135.front.sepia.ceph.com. reason: Cannot unlock an already-unlocked node
2024-04-10T19:37:47.558 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:47.588 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:47.600 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:47.671 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is on
2024-04-10T19:37:47.699 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off
2024-04-10T19:37:47.711 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on
2024-04-10T19:37:47.800 INFO:teuthology.orchestra.console.smithi067:Power off completed
2024-04-10T19:37:47.987 INFO:teuthology.lock.ops:unlocked: smithi067.front.sepia.ceph.com
2024-04-10T19:37:51.676 DEBUG:teuthology.orchestra.console.smithi047:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:51.712 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-04-10T19:37:51.786 DEBUG:teuthology.orchestra.console.smithi047:check power output: Chassis Power is off
2024-04-10T19:37:51.821 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off
2024-04-10T19:37:51.888 INFO:teuthology.orchestra.console.smithi047:Power off completed
2024-04-10T19:37:51.922 INFO:teuthology.orchestra.console.smithi170:Power off completed
2024-04-10T19:37:52.209 INFO:teuthology.lock.ops:unlocked: smithi047.front.sepia.ceph.com
2024-04-10T19:37:52.220 INFO:teuthology.lock.ops:unlocked: smithi170.front.sepia.ceph.com
2024-04-10T19:37:52.743 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/supervisor.py", line 58, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/dispatcher/supervisor.py", line 232, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 326, in reimage_machines
    reimaged = do_update_keys(list(reimaged.keys()))[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/lock/ops.py", line 290, in do_update_keys
    keys_dict = misc.ssh_keyscan(machines, _raise=_raise)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1108, in ssh_keyscan
    key = _ssh_keyscan(hostname)
  File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/misc.py", line 1145, in _ssh_keyscan
    return sorted(keys)[0]
IndexError: list index out of range