2024-02-14T18:00:47.811 INFO:root:teuthology version: 0.0.1.dev258+g9e44c16 2024-02-14T18:00:47.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T18:00:47.930 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5008b2ac10>, signals=[15]) 2024-02-14T18:00:47.930 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging 2024-02-14T18:00:47.930 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server 2024-02-14T18:00:47.931 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-02-14T18:00:48.049 DEBUG:teuthology.provision.fog.smithi072:Requesting image smithi_centos_9.stream (ID 10) 2024-02-14T18:00:48.084 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of centos 9.stream 2024-02-14T18:00:48.357 INFO:teuthology.orchestra.console.smithi072:Power off 2024-02-14T18:00:48.357 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T18:00:48.379 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2024-02-14T18:00:48.379 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:00:48.399 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-14T18:00:48.500 INFO:teuthology.orchestra.console.smithi072:Power off completed 2024-02-14T18:00:48.601 INFO:teuthology.orchestra.console.smithi072:Power on 2024-02-14T18:00:48.601 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-14T18:00:48.622 DEBUG:teuthology.orchestra.console.smithi072:power on output: Chassis Power Control: Up/On 2024-02-14T18:00:48.622 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:00:48.642 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-14T18:00:52.647 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:00:52.756 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-14T18:00:52.856 INFO:teuthology.orchestra.console.smithi072:Power on completed 2024-02-14T18:00:52.957 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2024-02-14T18:03:54.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:03:57.724 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:04:03.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:04:06.780 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:04:12.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:04:15.872 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:04:21.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:04:24.960 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:04:30.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:04:34.044 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:04:40.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:04:43.100 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:04:49.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:04:52.156 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:04:58.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:05:16.476 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:05:22.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:05:25.564 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:05:31.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:05:34.652 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:05:40.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:05:43.708 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:05:49.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:05:52.796 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:05:58.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:06:01.852 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:06:07.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:06:10.912 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:06:16.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:06:19.996 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:06:26.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:06:29.052 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:06:35.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:06:38.144 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:06:44.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:06:47.228 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:06:53.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:06:56.284 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:07:02.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:07:05.376 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:07:11.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:07:14.460 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:07:20.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:07:23.516 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:07:29.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:07:32.604 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:07:38.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:07:41.660 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:07:47.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:07:50.748 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:07:56.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:07:59.804 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:08:05.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:08:08.896 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:08:14.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:08:17.980 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:08:23.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:08:27.068 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:08:33.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:08:36.124 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:08:42.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:08:45.180 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:08:51.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:08:54.268 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:09:00.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:09:03.356 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:09:09.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:09:12.444 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:09:18.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:09:21.500 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:09:27.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:09:30.588 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:09:36.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:09:39.648 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:09:45.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:09:48.732 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:09:54.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:09:57.792 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:10:03.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:10:06.876 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:10:12.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:10:15.932 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:10:21.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:10:24.988 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:10:30.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:10:34.044 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:10:40.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:10:43.132 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:10:49.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:10:52.188 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:10:58.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:11:01.244 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:11:07.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:11:10.332 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:11:16.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:11:19.388 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:11:25.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:11:28.448 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:11:34.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:11:37.536 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:11:43.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:11:46.620 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:11:52.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:11:55.680 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:12:01.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:12:04.764 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:12:10.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:12:13.820 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:12:19.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:12:22.876 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:12:28.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:12:31.935 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:12:37.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:12:41.024 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:12:47.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:12:50.108 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:12:56.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:12:59.196 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:13:05.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:13:08.284 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:13:14.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:13:17.340 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:13:23.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:13:26.396 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:13:32.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:13:35.456 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:13:41.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:13:44.540 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:13:50.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:13:53.600 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:13:59.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:14:02.688 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:14:08.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:14:11.772 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:14:17.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:14:20.860 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:14:26.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:14:29.948 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:14:35.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:14:39.008 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:14:45.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:14:48.092 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:14:54.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:14:57.148 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:15:03.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:15:06.204 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:15:12.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:15:15.260 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:15:21.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:15:24.316 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:15:30.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:15:33.372 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:15:39.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:15:42.460 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:15:48.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:15:51.548 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:15:57.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:16:00.636 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:16:06.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:16:09.692 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:16:15.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:16:18.752 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:16:24.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:16:27.836 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:16:33.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:16:36.892 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:16:42.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:16:45.952 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:16:51.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:16:55.036 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:17:01.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:17:04.128 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:17:10.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:17:13.212 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:17:19.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:17:22.268 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:17:28.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:17:31.324 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:17:37.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:17:40.384 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:17:46.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:17:49.468 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:17:55.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:17:58.532 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:18:04.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:18:07.616 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:18:13.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:18:16.700 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:18:22.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:18:25.756 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:18:31.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:18:34.815 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:18:40.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:18:43.900 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:18:49.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:18:52.960 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:18:58.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:19:02.044 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:19:08.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T18:19:11.137 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T18:19:11.139 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-14T18:19:11.140 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T18:19:11.458 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=19fdd4fe0aa946afa6ff1019b2d2c775 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-14T18:19:11.460 INFO:teuthology.nuke:Checking targets against current locks 2024-02-14T18:19:11.530 DEBUG:teuthology.nuke:shortname: smithi072 2024-02-14T18:19:11.530 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-14T18:19:11.554 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-13_23:40:47-rados-wip-yuri-testing-2024-02-13-0903-distro-default-smithi/7558917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-14 18:00:46.802902', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNzBOX8ek7SXeZ1GpsbAzpRvrmsWDmV4UYiSzQgg+nA7znm7woBDcogAuOF2s1fW4Gm9RdcGpsjHfVdJiMb+ut0='} 2024-02-14T18:19:11.604 INFO:teuthology.orchestra.console.smithi072:Power off 2024-02-14T18:19:11.605 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T18:19:11.628 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2024-02-14T18:19:11.629 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:19:11.651 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-14T18:19:15.656 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:19:15.765 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-14T18:19:19.768 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T18:19:19.877 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-14T18:19:19.978 INFO:teuthology.orchestra.console.smithi072:Power off completed 2024-02-14T18:19:20.139 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com 2024-02-14T18:19:20.265 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds