2024-02-17T09:59:58.530 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-17T09:59:58.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-17T09:59:58.681 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd85490ec10>, signals=[15]) 2024-02-17T09:59:58.682 INFO:teuthology.lock.ops:Start node 'smithi083.front.sepia.ceph.com' reimaging 2024-02-17T09:59:58.682 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: reset os type and version on server 2024-02-17T09:59:58.682 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-02-17T09:59:58.712 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging 2024-02-17T09:59:58.713 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server 2024-02-17T09:59:58.713 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2024-02-17T09:59:58.835 DEBUG:teuthology.provision.fog.smithi083:Requesting image smithi_centos_9.stream (ID 10) 2024-02-17T09:59:58.843 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_centos_9.stream (ID 10) 2024-02-17T09:59:58.877 INFO:teuthology.provision.fog.smithi083:Scheduling deploy of centos 9.stream 2024-02-17T09:59:58.890 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 9.stream 2024-02-17T09:59:59.090 INFO:teuthology.orchestra.console.smithi083:Power off 2024-02-17T09:59:59.090 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T09:59:59.103 INFO:teuthology.orchestra.console.smithi140:Power off 2024-02-17T09:59:59.103 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T09:59:59.111 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-02-17T09:59:59.112 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T09:59:59.127 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2024-02-17T09:59:59.127 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T09:59:59.136 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-02-17T09:59:59.149 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-17T09:59:59.237 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-02-17T09:59:59.250 INFO:teuthology.orchestra.console.smithi140:Power off completed 2024-02-17T09:59:59.338 INFO:teuthology.orchestra.console.smithi083:Power on 2024-02-17T09:59:59.338 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-17T09:59:59.350 INFO:teuthology.orchestra.console.smithi140:Power on 2024-02-17T09:59:59.351 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-17T09:59:59.360 DEBUG:teuthology.orchestra.console.smithi083:power on output: Chassis Power Control: Up/On 2024-02-17T09:59:59.361 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T09:59:59.372 DEBUG:teuthology.orchestra.console.smithi140:power on output: Chassis Power Control: Up/On 2024-02-17T09:59:59.372 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T09:59:59.382 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-02-17T09:59:59.393 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-17T10:00:03.384 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:00:03.393 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:00:03.493 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-02-17T10:00:03.502 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-17T10:00:03.603 INFO:teuthology.orchestra.console.smithi140:Power on completed 2024-02-17T10:00:03.703 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2024-02-17T10:00:07.500 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:00:07.610 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-02-17T10:00:07.710 INFO:teuthology.orchestra.console.smithi083:Power on completed 2024-02-17T10:00:07.811 INFO:teuthology.provision.fog.smithi083:Waiting for deploy to finish 2024-02-17T10:02:38.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:02:41.500 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-02-17T10:02:47.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:02:49.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:02:50.556 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-02-17T10:02:56.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:02:59.644 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-02-17T10:03:05.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:07.968 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:03:08.700 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-02-17T10:03:13.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:14.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:17.054 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:03:17.760 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-02-17T10:03:23.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:23.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:26.140 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:03:26.812 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-02-17T10:03:32.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:32.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:35.228 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:03:35.868 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-02-17T10:03:41.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:41.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:44.284 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:03:50.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:03:53.340 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:03:59.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:04:41.914 WARNING:teuthology.provision.fog.smithi083:timed out 2024-02-17T10:04:47.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:04:48.210 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-17T10:04:48.211 DEBUG:teuthology.orchestra.run.smithi083:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-17T10:04:48.535 INFO:teuthology.provision.fog.smithi083:Node is ready 2024-02-17T10:04:48.536 DEBUG:teuthology.orchestra.run.smithi083:> hostname 2024-02-17T10:04:48.550 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083.front.sepia.ceph.com 2024-02-17T10:04:48.551 DEBUG:teuthology.orchestra.run.smithi083:> grep smithi083.front.sepia.ceph.com /etc/hosts 2024-02-17T10:04:48.607 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83 smithi083.front.sepia.ceph.com smithi083 2024-02-17T10:04:48.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname smithi083 2024-02-17T10:04:48.713 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i -e 's/smithi083.front.sepia.ceph.com/smithi083/g' /etc/hosts 2024-02-17T10:04:48.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i -e 's/172.21.15.83/172.21.15.83/g' /etc/hosts 2024-02-17T10:04:48.809 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i -e 's/smithi083.front.sepia.ceph.com/smithi083/g' /etc/hostname 2024-02-17T10:04:48.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname smithi083 2024-02-17T10:04:48.943 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-02-17T10:04:48.998 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-02-17T10:04:48.999 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-17T10:04:48.999 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-02-17T10:04:48.999 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-17T10:04:48.999 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-17T10:04:48.999 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-17T10:04:48.999 INFO:teuthology.provision.fog.smithi083:Deploy complete! 2024-02-17T10:04:49.001 DEBUG:teuthology.parallel:result is None 2024-02-17T10:04:59.356 WARNING:teuthology.provision.fog.smithi140:timed out 2024-02-17T10:05:05.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:05:23.904 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:05:29.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:05:32.992 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:05:39.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:05:42.076 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:05:48.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:05:51.132 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:05:57.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:06:00.220 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:06:06.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:06:09.308 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:06:15.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:06:18.396 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:06:24.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:06:27.452 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:06:33.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:06:36.508 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:06:42.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:06:45.600 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:06:51.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:06:54.684 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:07:00.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:07:03.740 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:07:09.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:07:12.828 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:07:18.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:07:21.916 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:07:27.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:07:31.008 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:07:37.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:07:40.092 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:07:46.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:07:49.180 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:07:55.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:07:58.236 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:08:04.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:08:07.324 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:08:13.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:08:16.380 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:08:22.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:08:25.436 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:08:31.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:08:34.492 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:08:40.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:08:43.580 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:08:49.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:08:52.636 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:08:58.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:09:01.724 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:09:07.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:09:10.780 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:09:16.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:09:19.868 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:09:25.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:09:28.924 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:09:34.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:09:38.016 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:09:44.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:09:47.100 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:09:53.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:09:56.156 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:10:02.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:10:05.212 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:10:11.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:10:14.300 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:10:20.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:10:23.356 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:10:29.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:10:32.444 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:10:38.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:10:41.532 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:10:47.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:10:50.592 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:10:56.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:10:59.676 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:11:05.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:11:08.764 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:11:14.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:11:17.820 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:11:23.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:11:26.908 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:11:32.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:11:35.964 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:11:41.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:11:45.020 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:11:51.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:11:54.079 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:12:00.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:12:03.164 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:12:09.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:12:12.220 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:12:18.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:12:21.308 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:12:27.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:12:30.396 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:12:36.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:12:39.484 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:12:45.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:12:48.540 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:12:54.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:12:57.596 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:13:03.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:13:06.684 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:13:12.685 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:13:15.740 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:13:21.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:13:24.828 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:13:30.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:13:33.916 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:13:39.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:13:43.008 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:13:49.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:13:52.095 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:13:58.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:14:01.180 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:14:07.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:14:10.240 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:14:16.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:14:19.324 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:14:25.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:14:28.412 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:14:34.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:14:37.468 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:14:43.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:14:46.524 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:14:52.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:14:55.580 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:15:01.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:15:04.636 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:15:10.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:15:13.692 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:15:19.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:15:22.748 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:15:28.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:15:31.836 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:15:37.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:15:40.928 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:15:46.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:15:50.012 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:15:56.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:15:59.068 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:16:05.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:16:08.124 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:16:14.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:16:17.180 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:16:23.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:16:26.236 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:16:32.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:16:35.324 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:16:41.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:16:44.380 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:16:50.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:16:53.468 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:16:59.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:17:02.524 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:17:08.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:17:11.580 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:17:17.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:17:20.636 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:17:26.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:17:29.724 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:17:35.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:17:38.780 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:17:44.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:17:47.836 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:17:53.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:17:56.924 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:18:02.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:18:06.012 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:18:12.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:18:15.100 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:18:21.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:18:24.188 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:18:30.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:18:33.276 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:18:39.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:18:42.364 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:18:48.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:18:51.452 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:18:57.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:19:00.508 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:19:06.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:19:09.568 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:19:15.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T10:19:18.652 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-17T10:19:18.654 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-17T10:19:18.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-17T10:19:18.934 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=50638d8b16ce4b36883024f3fb763cc0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-17T10:19:18.936 INFO:teuthology.nuke:Checking targets against current locks 2024-02-17T10:19:19.007 DEBUG:teuthology.nuke:shortname: smithi083 2024-02-17T10:19:19.007 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T10:19:19.030 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:14:07-orch-main-distro-default-smithi/7563047', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 09:59:57.490291', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMbbwfqlyXJ1M0qHUT8eZvjMUmziwAI4fV/Wf2IDggmG5KWw+KHAKlXu6CGIHrn4OLIEvS+EvDYKbeNNBVoeIPM='} 2024-02-17T10:19:19.033 DEBUG:teuthology.nuke:shortname: smithi140 2024-02-17T10:19:19.033 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T10:19:19.056 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:14:07-orch-main-distro-default-smithi/7563047', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 09:59:57.488542', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOb7w5wu3h2313zeOkPW157UIcoXbTIZtz+iKADKtPG4jQRbj6TGceiKVlxCSU79bcqLc6ffV6QvEAjkg6h/6rA='} 2024-02-17T10:19:19.082 INFO:teuthology.orchestra.console.smithi083:Power off 2024-02-17T10:19:19.082 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T10:19:19.105 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-02-17T10:19:19.105 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:19:19.122 INFO:teuthology.orchestra.console.smithi140:Power off 2024-02-17T10:19:19.122 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T10:19:19.131 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-02-17T10:19:19.145 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2024-02-17T10:19:19.145 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:19:19.169 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-17T10:19:23.136 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:19:23.171 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:19:23.245 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-02-17T10:19:23.279 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-17T10:19:27.250 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:19:27.280 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T10:19:27.363 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-02-17T10:19:27.388 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-17T10:19:27.465 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-02-17T10:19:27.489 INFO:teuthology.orchestra.console.smithi140:Power off completed 2024-02-17T10:19:27.627 INFO:teuthology.lock.ops:unlocked: smithi083.front.sepia.ceph.com 2024-02-17T10:19:27.636 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2024-02-17T10:19:27.865 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds