2024-02-08T06:05:56.530 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-08T06:05:56.530 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T06:06:16.622 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fee72228c10>, signals=[15]) 2024-02-08T06:06:16.622 INFO:teuthology.lock.ops:Start node 'smithi059.front.sepia.ceph.com' reimaging 2024-02-08T06:06:16.623 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: reset os type and version on server 2024-02-08T06:06:16.624 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-02-08T06:06:20.747 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging 2024-02-08T06:06:20.747 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server 2024-02-08T06:06:20.747 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-02-08T06:06:30.869 DEBUG:teuthology.provision.fog.smithi059:Requesting image smithi_centos_9.stream (ID 10) 2024-02-08T06:06:30.872 DEBUG:teuthology.provision.fog.smithi072:Requesting image smithi_centos_9.stream (ID 10) 2024-02-08T06:06:30.917 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of centos 9.stream 2024-02-08T06:06:30.920 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of centos 9.stream 2024-02-08T06:06:31.165 INFO:teuthology.orchestra.console.smithi072:Power off 2024-02-08T06:06:31.165 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-08T06:06:31.180 INFO:teuthology.orchestra.console.smithi059:Power off 2024-02-08T06:06:31.180 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-08T06:06:31.188 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2024-02-08T06:06:31.188 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:31.201 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2024-02-08T06:06:31.201 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:31.211 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-08T06:06:31.230 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-02-08T06:06:31.312 INFO:teuthology.orchestra.console.smithi072:Power off completed 2024-02-08T06:06:31.331 INFO:teuthology.orchestra.console.smithi059:Power off completed 2024-02-08T06:06:31.413 INFO:teuthology.orchestra.console.smithi072:Power on 2024-02-08T06:06:31.413 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-08T06:06:31.432 INFO:teuthology.orchestra.console.smithi059:Power on 2024-02-08T06:06:31.432 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-08T06:06:31.441 DEBUG:teuthology.orchestra.console.smithi072:power on output: Chassis Power Control: Up/On 2024-02-08T06:06:31.442 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:31.455 DEBUG:teuthology.orchestra.console.smithi059:power on output: Chassis Power Control: Up/On 2024-02-08T06:06:31.455 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:31.464 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-08T06:06:31.477 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-02-08T06:06:35.469 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:35.479 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:36.578 DEBUG:teuthology.orchestra.console.smithi072:check power output: 2024-02-08T06:06:36.587 DEBUG:teuthology.orchestra.console.smithi059:check power output: 2024-02-08T06:06:40.579 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:40.588 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:41.690 DEBUG:teuthology.orchestra.console.smithi072:check power output: 2024-02-08T06:06:41.696 DEBUG:teuthology.orchestra.console.smithi059:check power output: 2024-02-08T06:06:45.693 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:45.701 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:46.804 DEBUG:teuthology.orchestra.console.smithi072:check power output: 2024-02-08T06:06:46.809 DEBUG:teuthology.orchestra.console.smithi059:check power output: 2024-02-08T06:06:50.806 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:50.812 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:06:50.914 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-08T06:06:50.920 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-02-08T06:06:51.016 INFO:teuthology.orchestra.console.smithi072:Power on completed 2024-02-08T06:06:51.021 INFO:teuthology.orchestra.console.smithi059:Power on completed 2024-02-08T06:06:51.117 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2024-02-08T06:07:01.126 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2024-02-08T06:09:21.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:09:25.052 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-02-08T06:09:31.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:09:34.140 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-02-08T06:09:37.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:09:40.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:09:45.151 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:09:48.220 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-02-08T06:09:51.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:09:54.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:09:54.236 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:09:57.276 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-02-08T06:10:00.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:10:03.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:10:08.287 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:10:11.356 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-02-08T06:10:14.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:10:19.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:10:24.304 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:10:30.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:10:33.372 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:10:39.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:10:55.953 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-08T06:10:55.953 DEBUG:teuthology.orchestra.run.smithi059:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-08T06:11:02.844 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:11:06.330 INFO:teuthology.provision.fog.smithi059:Node is ready 2024-02-08T06:11:06.330 DEBUG:teuthology.orchestra.run.smithi059:> hostname 2024-02-08T06:11:06.349 INFO:teuthology.orchestra.run.smithi059.stdout:smithi046 2024-02-08T06:11:06.349 DEBUG:teuthology.orchestra.run.smithi059:> grep smithi046 /etc/hosts 2024-02-08T06:11:06.407 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-02-08T06:11:06.408 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2024-02-08T06:11:06.564 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi046/smithi059/g' /etc/hosts 2024-02-08T06:11:06.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/172.21.15.46/172.21.15.59/g' /etc/hosts 2024-02-08T06:11:06.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi046/smithi059/g' /etc/hostname 2024-02-08T06:11:06.745 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2024-02-08T06:11:06.818 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-02-08T06:11:06.874 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-02-08T06:11:06.874 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-02-08T06:11:06.874 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-02-08T06:11:06.874 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-08T06:11:06.875 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-08T06:11:06.876 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2024-02-08T06:11:06.878 DEBUG:teuthology.parallel:result is None 2024-02-08T06:11:08.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:11:16.924 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:11:22.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:11:31.004 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:11:37.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:11:41.372 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:11:47.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:11:50.428 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:11:56.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:12:04.508 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:12:10.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:12:13.596 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:12:19.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:12:22.684 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:12:28.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:12:31.740 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:12:37.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:12:40.796 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:12:46.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:12:54.876 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:13:00.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:13:03.964 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:13:09.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:13:18.048 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:13:24.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:13:32.125 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:13:38.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:13:42.364 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:13:48.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:13:51.456 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:13:57.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:14:05.532 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:14:11.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:14:19.612 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:14:25.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:14:30.108 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:14:36.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:14:44.188 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:14:50.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:14:53.244 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:14:59.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:15:02.300 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:15:08.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:15:11.360 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:15:17.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:15:20.444 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:15:26.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:15:34.524 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:15:40.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:15:43.580 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:15:49.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:15:57.660 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:16:03.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:16:06.717 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:16:12.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:16:15.804 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:16:21.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:16:29.884 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:16:35.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:16:43.964 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:16:49.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:16:54.460 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:17:00.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:17:03.548 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:17:09.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:17:17.628 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:17:23.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:17:31.708 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:17:37.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:17:43.612 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:17:49.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:17:52.700 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:17:58.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:18:06.780 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:18:12.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:18:20.860 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:18:26.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:18:34.944 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:18:40.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:18:44.028 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:18:50.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:18:53.116 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:18:59.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:19:07.200 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:19:13.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:19:16.284 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:19:22.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:19:30.364 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:19:36.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:19:44.448 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:19:50.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:19:54.880 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:20:00.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:20:03.967 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:20:09.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:20:18.044 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:20:24.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:20:32.124 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:20:38.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:20:41.180 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:20:47.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:20:50.240 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:20:56.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:20:59.328 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:21:05.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:21:08.412 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:21:14.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:21:22.492 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:21:28.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:21:31.548 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:21:37.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:21:40.636 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:21:46.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:21:49.692 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:21:55.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:21:58.748 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:22:04.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:22:07.804 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:22:13.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:22:16.892 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:22:22.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:22:25.952 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:22:31.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:22:35.036 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:22:41.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:22:49.116 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:22:55.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:23:03.196 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:23:09.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:23:12.284 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:23:18.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:23:26.364 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:23:32.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:23:35.420 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:23:41.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:23:49.500 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:23:55.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:24:03.580 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:24:09.582 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:24:12.636 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:24:18.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:24:26.716 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:24:32.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:24:40.800 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:24:46.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:24:54.876 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:25:00.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:25:08.956 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:25:14.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:25:23.036 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:25:29.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:25:32.092 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:25:38.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:25:46.172 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:25:52.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:25:55.260 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:26:01.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:26:09.340 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:26:15.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:26:23.420 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:26:29.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:26:37.500 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:26:43.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:26:46.556 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:26:52.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:26:55.644 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:27:01.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:27:09.724 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:27:15.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:27:18.812 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:27:24.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:27:32.892 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:27:38.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:27:41.980 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:27:47.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:27:56.060 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:28:02.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:28:10.140 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:28:16.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:28:19.228 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:28:25.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:28:33.308 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:28:39.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:28:42.396 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:28:48.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:28:51.452 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:28:57.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-08T06:29:05.532 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-08T06:29:05.535 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-08T06:29:05.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-08T06:29:06.736 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=059ff05621424eb892d997cceafd61fc Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-08T06:29:06.738 INFO:teuthology.nuke:Checking targets against current locks 2024-02-08T06:29:06.805 DEBUG:teuthology.nuke:shortname: smithi059 2024-02-08T06:29:06.805 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-08T06:29:06.819 DEBUG:teuthology.nuke:shortname: smithi072 2024-02-08T06:29:06.820 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-08T06:29:06.826 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-02-08_02:25:06-upgrade:reef-x-main-distro-default-smithi/7551525', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-08 06:05:55.520591', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLdXfUQDQ6d12uJlCgrR7gdZeunYSWV6YZqqgcOZmn0D5mmg8igTsd3CY0a6VLzdPnyRCAWzs+Qb3BDo1pT9fBg='} 2024-02-08T06:29:06.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-02-08_02:25:06-upgrade:reef-x-main-distro-default-smithi/7551525', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-08 06:05:55.522454', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIxnB3/4tdcaBF8FfoJn6HDJLxeP2xokirHHPb6zuuyKINB1qgPJo3XG/t3YEgdGW3pPzqGw+IeP3BKALC4nP0U='} 2024-02-08T06:29:11.876 INFO:teuthology.orchestra.console.smithi059:Power off 2024-02-08T06:29:11.876 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-08T06:29:11.899 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2024-02-08T06:29:11.899 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:11.917 INFO:teuthology.orchestra.console.smithi072:Power off 2024-02-08T06:29:11.917 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-08T06:29:11.925 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-02-08T06:29:11.937 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2024-02-08T06:29:11.938 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:11.959 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-08T06:29:15.928 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:15.960 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:16.038 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-02-08T06:29:16.068 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-08T06:29:20.040 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:20.069 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:20.179 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-08T06:29:20.280 INFO:teuthology.orchestra.console.smithi072:Power off completed 2024-02-08T06:29:20.430 INFO:teuthology.lock.ops:unlocked: smithi072.front.sepia.ceph.com 2024-02-08T06:29:21.151 DEBUG:teuthology.orchestra.console.smithi059:check power output: 2024-02-08T06:29:25.154 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:26.263 DEBUG:teuthology.orchestra.console.smithi059:check power output: 2024-02-08T06:29:30.264 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:31.373 DEBUG:teuthology.orchestra.console.smithi059:check power output: 2024-02-08T06:29:35.378 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:36.488 DEBUG:teuthology.orchestra.console.smithi059:check power output: 2024-02-08T06:29:40.489 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:41.597 DEBUG:teuthology.orchestra.console.smithi059:check power output: 2024-02-08T06:29:45.599 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-08T06:29:45.708 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-02-08T06:29:45.809 INFO:teuthology.orchestra.console.smithi059:Power off completed 2024-02-08T06:29:55.970 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2024-02-08T06:30:01.219 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds