2024-01-05T16:39:08.939 INFO:root:teuthology version: 0.0.1.dev228+g01dd171 2024-01-05T16:39:08.939 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-05T16:39:09.061 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff223cc4940>, signals=[15]) 2024-01-05T16:39:09.061 INFO:teuthology.lock.ops:Start node 'smithi059.front.sepia.ceph.com' reimaging 2024-01-05T16:39:09.061 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: reset os type and version on server 2024-01-05T16:39:09.061 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-01-05T16:39:09.096 INFO:teuthology.lock.ops:Start node 'smithi155.front.sepia.ceph.com' reimaging 2024-01-05T16:39:09.096 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: reset os type and version on server 2024-01-05T16:39:09.096 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-01-05T16:39:09.193 DEBUG:teuthology.provision.fog.smithi059:Requesting image smithi_centos_9.stream (ID 10) 2024-01-05T16:39:09.223 DEBUG:teuthology.provision.fog.smithi155:Requesting image smithi_centos_9.stream (ID 10) 2024-01-05T16:39:09.227 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of centos 9.stream 2024-01-05T16:39:09.264 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of centos 9.stream 2024-01-05T16:39:09.463 INFO:teuthology.orchestra.console.smithi059:Power off 2024-01-05T16:39:09.463 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-05T16:39:09.484 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2024-01-05T16:39:09.485 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-05T16:39:09.499 INFO:teuthology.orchestra.console.smithi155:Power off 2024-01-05T16:39:09.499 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-05T16:39:09.509 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-01-05T16:39:09.519 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2024-01-05T16:39:09.519 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-05T16:39:09.540 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-01-05T16:39:09.610 INFO:teuthology.orchestra.console.smithi059:Power off completed 2024-01-05T16:39:09.641 INFO:teuthology.orchestra.console.smithi155:Power off completed 2024-01-05T16:39:09.711 INFO:teuthology.orchestra.console.smithi059:Power on 2024-01-05T16:39:09.712 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-05T16:39:09.734 DEBUG:teuthology.orchestra.console.smithi059:power on output: Chassis Power Control: Up/On 2024-01-05T16:39:09.734 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-05T16:39:09.742 INFO:teuthology.orchestra.console.smithi155:Power on 2024-01-05T16:39:09.742 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-05T16:39:09.757 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-01-05T16:39:09.764 DEBUG:teuthology.orchestra.console.smithi155:power on output: Chassis Power Control: Up/On 2024-01-05T16:39:09.764 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-05T16:39:09.786 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-01-05T16:39:13.760 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-05T16:39:13.787 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-05T16:39:13.869 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-01-05T16:39:13.896 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-01-05T16:39:13.996 INFO:teuthology.orchestra.console.smithi155:Power on completed 2024-01-05T16:39:14.097 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2024-01-05T16:39:17.874 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-05T16:39:17.983 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-01-05T16:39:18.083 INFO:teuthology.orchestra.console.smithi059:Power on completed 2024-01-05T16:39:18.184 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2024-01-05T16:41:49.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:41:52.348 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:41:58.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:00.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:01.436 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:42:03.292 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:42:07.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:09.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:10.492 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:42:12.348 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:42:16.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:18.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:19.548 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:42:21.404 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:42:25.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:27.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:28.604 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:42:30.460 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:42:34.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:36.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:37.660 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:42:39.516 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:42:43.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:45.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:46.716 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:42:48.572 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:42:52.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:54.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:42:55.772 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:42:57.628 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:43:01.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:03.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:04.828 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:43:06.684 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:43:10.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:12.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:13.884 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:43:15.740 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:43:19.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:21.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:22.940 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:43:24.796 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:43:28.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:30.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:31.996 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:43:33.852 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:43:38.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:39.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:41.052 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:43:42.908 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:43:47.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:48.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:50.108 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:43:51.967 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:43:56.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:57.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:43:59.164 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:44:01.052 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:44:05.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:07.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:08.220 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:44:10.108 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:44:14.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:16.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:17.276 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:44:19.164 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:44:23.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:25.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:26.336 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:44:28.220 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:44:32.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:34.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:35.420 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:44:37.276 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:44:41.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:43.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:44.476 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:44:46.332 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:44:50.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:52.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:44:53.532 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:44:55.388 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:44:59.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:01.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:02.588 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:45:04.444 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:45:08.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:08.764 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:45:10.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:10.588 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:45:14.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:16.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:17.820 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:45:19.644 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:45:23.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:25.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:26.876 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:45:28.700 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:45:32.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:34.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:34.844 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:45:35.932 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:45:40.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:41.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:43.900 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:45:44.988 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:45:49.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:50.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:52.956 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:45:54.044 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:45:58.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:45:59.100 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:46:00.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:46:05.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:46:08.156 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:46:14.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:46:32.512 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:46:38.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:46:39.516 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:46:45.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:46:45.647 ERROR:teuthology.orchestra.connection:Error authenticating with smithi155.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:46:49.773 ERROR:teuthology.orchestra.connection:Error authenticating with smithi155.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:46:56.897 ERROR:teuthology.orchestra.connection:Error authenticating with smithi155.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:47:00.048 WARNING:teuthology.provision.fog.smithi059:timed out 2024-01-05T16:47:06.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:47:06.137 ERROR:teuthology.orchestra.connection:Error authenticating with smithi059.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:47:06.982 ERROR:teuthology.orchestra.connection:Error authenticating with smithi155.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:47:10.266 ERROR:teuthology.orchestra.connection:Error authenticating with smithi059.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:47:17.392 ERROR:teuthology.orchestra.connection:Error authenticating with smithi059.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:47:20.109 ERROR:teuthology.orchestra.connection:Error authenticating with smithi155.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:47:27.522 ERROR:teuthology.orchestra.connection:Error authenticating with smithi059.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:47:36.233 ERROR:teuthology.orchestra.connection:Error authenticating with smithi155.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-05T16:48:13.628 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:48:15.164 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:48:19.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:48:21.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:48:22.428 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:48:23.964 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-01-05T16:48:28.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:48:29.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:48:31.484 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:48:37.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:48:37.628 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:48:43.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:48:46.684 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:48:52.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:49:11.228 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:49:17.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:49:18.236 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-01-05T16:49:24.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:49:27.470 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-05T16:49:27.471 DEBUG:teuthology.orchestra.run.smithi155:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-05T16:49:29.966 WARNING:teuthology.provision.fog.smithi059:timed out 2024-01-05T16:49:35.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-05T16:49:36.249 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-05T16:49:36.249 DEBUG:teuthology.orchestra.run.smithi059:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-05T16:49:36.519 INFO:teuthology.provision.fog.smithi059:Node is ready 2024-01-05T16:49:36.519 DEBUG:teuthology.orchestra.run.smithi059:> hostname 2024-01-05T16:49:36.579 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2024-01-05T16:49:36.579 DEBUG:teuthology.orchestra.run.smithi059:> grep smithi059.front.sepia.ceph.com /etc/hosts 2024-01-05T16:49:36.640 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2024-01-05T16:49:36.640 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2024-01-05T16:49:36.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hosts 2024-01-05T16:49:36.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/172.21.15.59/172.21.15.59/g' /etc/hosts 2024-01-05T16:49:36.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hostname 2024-01-05T16:49:36.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2024-01-05T16:49:37.003 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-01-05T16:49:37.061 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 8" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-05T16:49:37.062 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-05T16:49:37.065 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/fog.py", line 347, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi059's OS to be centos 9 but found centos 8 2024-01-05T16:49:37.066 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-05T16:49:37.218 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ffd0c2c58e434fcbb79e273ec59a8de0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/fog.py", line 347, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi059's OS to be centos 9 but found centos 8 2024-01-05T16:49:37.220 INFO:teuthology.nuke:Checking targets against current locks 2024-01-05T16:49:37.255 INFO:teuthology.nuke:Not nuking smithi059.front.sepia.ceph.com because description doesn't match: lflores-2024-01-05_16:37:53-rados:thrash-wip-read-balancer-mgr-module-distro-default-smithi != /home/teuthworker/archive/lflores-2024-01-05_16:44:37-rados:thrash-old-clients-wip-read-balancer-mgr-module-distro-default-smithi/7508231 2024-01-05T16:49:37.275 INFO:teuthology.nuke:Not nuking smithi155.front.sepia.ceph.com because description doesn't match: lflores-2024-01-05_16:37:53-rados:thrash-wip-read-balancer-mgr-module-distro-default-smithi != /home/teuthworker/archive/lflores-2024-01-05_16:44:37-rados:thrash-old-clients-wip-read-balancer-mgr-module-distro-default-smithi/7508230 2024-01-05T16:49:37.276 ERROR:teuthology.nuke:Could not nuke the following targets: targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKGhXkrtBTwJPw66H9mQodOTYVjl9/PjyRFCzRkPKOh25PHc8oAvlyd/yWrHzh6ewQMZjf+f3eZhrbUblL3ESJs= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC1yzIP8pZhg+KnRMw5gSS1AQNf4jhvE5m16i/qNsbgiE9098GjUCi3Bd6X+0DvUAk7bgUqfM2tfrZidQ+ZuhMM= 2024-01-05T16:49:37.526 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/teuthology/provision/fog.py", line 347, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi059's OS to be centos 9 but found centos 8