2024-02-01T04:35:30.726 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-01T04:35:30.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-01T04:35:30.878 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2b54519c10>, signals=[15]) 2024-02-01T04:35:30.879 INFO:teuthology.lock.ops:Start node 'smithi008.front.sepia.ceph.com' reimaging 2024-02-01T04:35:30.879 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: reset os type and version on server 2024-02-01T04:35:30.879 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2024-02-01T04:35:30.907 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging 2024-02-01T04:35:30.907 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server 2024-02-01T04:35:30.908 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2024-02-01T04:35:31.042 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_rhel_8.6 (ID 5) 2024-02-01T04:35:31.045 DEBUG:teuthology.provision.fog.smithi008:Requesting image smithi_rhel_8.6 (ID 5) 2024-02-01T04:35:31.089 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of rhel 8.6 2024-02-01T04:35:31.092 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of rhel 8.6 2024-02-01T04:35:31.291 INFO:teuthology.orchestra.console.smithi008:Power off 2024-02-01T04:35:31.291 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-01T04:35:31.310 INFO:teuthology.orchestra.console.smithi140:Power off 2024-02-01T04:35:31.310 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-01T04:35:31.319 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2024-02-01T04:35:31.319 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:35:31.332 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2024-02-01T04:35:31.332 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:35:31.340 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-02-01T04:35:31.352 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-01T04:35:31.441 INFO:teuthology.orchestra.console.smithi008:Power off completed 2024-02-01T04:35:31.453 INFO:teuthology.orchestra.console.smithi140:Power off completed 2024-02-01T04:35:31.542 INFO:teuthology.orchestra.console.smithi008:Power on 2024-02-01T04:35:31.542 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-01T04:35:31.555 INFO:teuthology.orchestra.console.smithi140:Power on 2024-02-01T04:35:31.555 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-01T04:35:31.565 DEBUG:teuthology.orchestra.console.smithi008:power on output: Chassis Power Control: Up/On 2024-02-01T04:35:31.565 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:35:31.578 DEBUG:teuthology.orchestra.console.smithi140:power on output: Chassis Power Control: Up/On 2024-02-01T04:35:31.579 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:35:31.587 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-02-01T04:35:31.603 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-01T04:35:35.590 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:35:35.604 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:35:35.700 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-02-01T04:35:35.713 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-01T04:35:35.813 INFO:teuthology.orchestra.console.smithi140:Power on completed 2024-02-01T04:35:35.914 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2024-02-01T04:35:39.702 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:35:39.811 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-02-01T04:35:39.911 INFO:teuthology.orchestra.console.smithi008:Power on completed 2024-02-01T04:35:40.012 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish 2024-02-01T04:38:10.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:13.692 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-01T04:38:19.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:21.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:22.780 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-01T04:38:24.732 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:38:28.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:30.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:31.840 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-01T04:38:33.788 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:38:37.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:39.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:40.924 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-01T04:38:42.844 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:38:46.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:48.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:50.012 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-01T04:38:51.900 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:38:56.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:57.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:38:59.068 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-01T04:39:00.956 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:39:05.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:39:06.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:39:08.128 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-01T04:39:10.016 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:39:14.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:39:16.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:39:19.100 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:39:25.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:39:45.984 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-01T04:39:45.984 DEBUG:teuthology.orchestra.run.smithi008:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-01T04:40:05.927 INFO:teuthology.provision.fog.smithi008:Node is ready 2024-02-01T04:40:05.928 DEBUG:teuthology.orchestra.run.smithi008:> hostname 2024-02-01T04:40:05.951 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com 2024-02-01T04:40:05.951 DEBUG:teuthology.orchestra.run.smithi008:> grep smithi008.front.sepia.ceph.com /etc/hosts 2024-02-01T04:40:06.014 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008 2024-02-01T04:40:06.014 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname smithi008 2024-02-01T04:40:06.161 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/smithi008.front.sepia.ceph.com/smithi008/g' /etc/hosts 2024-02-01T04:40:06.195 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/172.21.15.8/172.21.15.8/g' /etc/hosts 2024-02-01T04:40:06.270 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/smithi008.front.sepia.ceph.com/smithi008/g' /etc/hostname 2024-02-01T04:40:06.344 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname smithi008 2024-02-01T04:40:06.419 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2024-02-01T04:40:06.479 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="Red Hat Enterprise Linux" 2024-02-01T04:40:06.479 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8.6 (Ootpa)" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:ID="rhel" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="fedora" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8.6" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.redhat.com/" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-01T04:40:06.480 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-02-01T04:40:06.481 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-02-01T04:40:06.481 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-02-01T04:40:06.481 INFO:teuthology.provision.fog.smithi008:Deploy complete! 2024-02-01T04:40:06.483 DEBUG:teuthology.parallel:result is None 2024-02-01T04:40:25.111 WARNING:teuthology.provision.fog.smithi140:timed out 2024-02-01T04:40:31.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:40:49.472 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:40:55.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:40:58.556 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:41:04.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:41:07.616 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:41:13.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:41:16.700 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:41:22.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:41:25.760 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:41:31.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:41:34.848 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:41:40.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:41:43.932 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:41:49.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:41:52.988 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:41:58.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:42:02.076 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:42:08.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:42:11.164 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:42:17.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:42:20.252 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:42:26.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:42:29.308 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:42:35.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:42:38.396 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:42:44.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:42:47.484 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:42:53.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:42:56.540 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:43:02.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:43:05.628 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:43:11.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:43:14.716 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:43:20.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:43:23.804 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:43:29.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:43:32.860 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:43:38.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:43:41.916 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:43:47.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:43:51.004 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:43:57.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:44:00.092 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:44:06.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:44:09.148 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:44:15.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:44:18.236 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:44:24.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:44:27.324 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:44:33.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:44:36.412 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:44:42.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:44:45.468 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:44:51.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:44:54.524 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:45:00.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:45:03.612 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:45:09.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:45:12.668 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:45:18.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:45:21.728 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:45:27.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:45:30.812 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:45:36.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:45:39.904 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:45:45.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:45:48.988 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:45:54.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:45:58.048 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:46:04.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:46:07.136 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:46:13.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:46:16.220 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:46:22.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:46:25.308 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:46:31.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:46:34.396 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:46:40.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:46:43.452 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:46:49.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:46:52.540 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:46:58.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:47:01.632 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:47:07.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:47:10.720 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:47:16.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:47:19.804 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:47:25.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:47:28.892 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:47:34.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:47:37.948 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:47:43.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:47:47.036 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:47:53.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:47:56.124 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:48:02.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:48:05.180 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:48:11.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:48:14.236 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:48:20.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:48:23.292 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:48:29.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:48:32.380 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:48:38.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:48:41.468 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:48:47.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:48:50.524 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:48:56.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:48:59.580 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:49:05.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:49:08.636 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:49:14.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:49:17.692 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:49:23.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:49:26.780 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:49:32.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:49:35.868 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:49:41.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:49:44.924 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:49:50.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:49:54.012 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:50:00.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:50:03.100 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:50:09.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:50:12.156 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:50:18.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:50:21.212 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:50:27.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:50:30.268 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:50:36.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:50:39.356 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:50:45.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:50:48.444 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:50:54.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:50:57.500 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:51:03.502 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:51:06.556 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:51:12.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:51:15.644 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:51:21.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:51:24.732 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:51:30.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:51:33.820 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:51:39.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:51:42.876 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:51:48.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:51:51.964 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:51:57.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:52:01.020 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:52:07.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:52:10.112 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:52:16.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:52:19.196 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:52:25.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:52:28.284 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:52:34.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:52:37.340 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:52:43.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:52:46.432 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:52:52.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:52:55.584 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:53:01.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:53:04.668 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:53:10.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:53:13.724 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:53:19.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:53:22.812 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:53:28.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:53:31.872 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:53:37.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:53:40.956 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:53:47.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:53:50.236 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:53:56.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:53:59.324 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:54:05.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:54:08.380 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:54:14.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:54:17.468 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:54:23.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:54:26.560 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:54:32.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-01T04:54:35.644 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-01T04:54:35.646 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-01T04:54:35.648 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-01T04:54:35.944 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=865fa4d4ab99434cb2a8b9628d161e96 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-01T04:54:35.946 INFO:teuthology.nuke:Checking targets against current locks 2024-02-01T04:54:36.017 DEBUG:teuthology.nuke:shortname: smithi008 2024-02-01T04:54:36.017 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-01T04:54:36.039 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-02-01_02:25:04-upgrade:reef-x-main-distro-default-smithi/7541642', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-01 04:35:29.659428', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiojIF71bfPi6zNXtAfzsRhTJctUZmcmzTWBnsMgt4gxBytyz0WsnsrQtVkOTkD99riwjpaW8QoBf7tHiakR4A='} 2024-02-01T04:54:36.043 DEBUG:teuthology.nuke:shortname: smithi140 2024-02-01T04:54:36.043 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-01T04:54:36.065 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-02-01_02:25:04-upgrade:reef-x-main-distro-default-smithi/7541642', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-01 04:35:29.657996', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2024-02-01T04:54:36.083 INFO:teuthology.orchestra.console.smithi008:Power off 2024-02-01T04:54:36.084 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-01T04:54:36.110 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2024-02-01T04:54:36.110 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:54:36.124 INFO:teuthology.orchestra.console.smithi140:Power off 2024-02-01T04:54:36.125 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-01T04:54:36.146 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2024-02-01T04:54:36.147 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:54:36.157 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-02-01T04:54:36.169 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-01T04:54:40.159 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:54:40.170 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:54:40.269 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-02-01T04:54:40.280 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-01T04:54:44.275 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:54:44.283 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-01T04:54:44.385 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-02-01T04:54:44.391 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-01T04:54:44.486 INFO:teuthology.orchestra.console.smithi008:Power off completed 2024-02-01T04:54:44.492 INFO:teuthology.orchestra.console.smithi140:Power off completed 2024-02-01T04:54:44.639 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2024-02-01T04:54:44.648 INFO:teuthology.lock.ops:unlocked: smithi008.front.sepia.ceph.com 2024-02-01T04:54:44.859 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds