2024-02-10T21:29:54.996 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-10T21:29:54.997 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T21:29:55.138 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6456789c10>, signals=[15]) 2024-02-10T21:29:55.138 INFO:teuthology.lock.ops:Start node 'smithi017.front.sepia.ceph.com' reimaging 2024-02-10T21:29:55.138 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: reset os type and version on server 2024-02-10T21:29:55.139 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-02-10T21:29:55.165 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging 2024-02-10T21:29:55.165 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server 2024-02-10T21:29:55.165 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2024-02-10T21:29:55.290 DEBUG:teuthology.provision.fog.smithi017:Requesting image smithi_centos_9.stream (ID 10) 2024-02-10T21:29:55.296 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_centos_9.stream (ID 10) 2024-02-10T21:29:55.331 INFO:teuthology.provision.fog.smithi017:Scheduling deploy of centos 9.stream 2024-02-10T21:29:55.340 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 9.stream 2024-02-10T21:29:55.548 INFO:teuthology.orchestra.console.smithi017:Power off 2024-02-10T21:29:55.549 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T21:29:55.563 INFO:teuthology.orchestra.console.smithi140:Power off 2024-02-10T21:29:55.564 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T21:29:55.573 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off 2024-02-10T21:29:55.573 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:29:55.585 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2024-02-10T21:29:55.585 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:29:55.596 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-02-10T21:29:55.606 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-10T21:29:55.707 INFO:teuthology.orchestra.console.smithi140:Power off completed 2024-02-10T21:29:55.808 INFO:teuthology.orchestra.console.smithi140:Power on 2024-02-10T21:29:55.808 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-10T21:29:55.829 DEBUG:teuthology.orchestra.console.smithi140:power on output: Chassis Power Control: Up/On 2024-02-10T21:29:55.830 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:29:55.851 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-10T21:29:59.600 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:29:59.710 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-02-10T21:29:59.852 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:29:59.961 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-10T21:30:03.713 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:30:03.822 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-02-10T21:30:03.922 INFO:teuthology.orchestra.console.smithi017:Power off completed 2024-02-10T21:30:03.963 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:30:04.024 INFO:teuthology.orchestra.console.smithi017:Power on 2024-02-10T21:30:04.024 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-10T21:30:04.045 DEBUG:teuthology.orchestra.console.smithi017:power on output: Chassis Power Control: Up/On 2024-02-10T21:30:04.045 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:30:04.066 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-02-10T21:30:04.161 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-10T21:30:04.262 INFO:teuthology.orchestra.console.smithi140:Power on completed 2024-02-10T21:30:04.363 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2024-02-10T21:30:08.068 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:30:08.177 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-02-10T21:30:12.180 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:30:12.290 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-02-10T21:30:12.391 INFO:teuthology.orchestra.console.smithi017:Power on completed 2024-02-10T21:30:12.491 INFO:teuthology.provision.fog.smithi017:Waiting for deploy to finish 2024-02-10T21:32:43.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:32:50.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:33:01.628 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-02-10T21:33:07.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:33:08.544 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:33:10.720 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-02-10T21:33:14.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:33:16.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:33:17.632 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:33:19.804 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-02-10T21:33:23.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:33:25.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:33:26.716 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:33:32.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:33:35.772 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:33:41.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:33:44.832 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:33:50.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:34:09.212 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:34:15.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:34:18.268 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:34:24.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:34:25.811 WARNING:teuthology.provision.fog.smithi017:timed out 2024-02-10T21:34:27.356 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:34:31.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:34:32.097 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-10T21:34:32.097 DEBUG:teuthology.orchestra.run.smithi017:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-10T21:34:32.445 INFO:teuthology.provision.fog.smithi017:Node is ready 2024-02-10T21:34:32.445 DEBUG:teuthology.orchestra.run.smithi017:> hostname 2024-02-10T21:34:32.461 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017.front.sepia.ceph.com 2024-02-10T21:34:32.461 DEBUG:teuthology.orchestra.run.smithi017:> grep smithi017.front.sepia.ceph.com /etc/hosts 2024-02-10T21:34:32.517 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17 smithi017.front.sepia.ceph.com smithi017 2024-02-10T21:34:32.517 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname smithi017 2024-02-10T21:34:32.624 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/smithi017.front.sepia.ceph.com/smithi017/g' /etc/hosts 2024-02-10T21:34:32.656 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/172.21.15.17/172.21.15.17/g' /etc/hosts 2024-02-10T21:34:32.727 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/smithi017.front.sepia.ceph.com/smithi017/g' /etc/hostname 2024-02-10T21:34:32.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname smithi017 2024-02-10T21:34:32.868 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-02-10T21:34:32.923 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream" 2024-02-10T21:34:32.923 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="9" 2024-02-10T21:34:32.923 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos" 2024-02-10T21:34:32.923 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora" 2024-02-10T21:34:32.923 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="9" 2024-02-10T21:34:32.923 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el9" 2024-02-10T21:34:32.923 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T21:34:32.924 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2024-02-10T21:34:32.924 INFO:teuthology.orchestra.run.smithi017.stdout:LOGO="fedora-logo-icon" 2024-02-10T21:34:32.924 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T21:34:32.924 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/" 2024-02-10T21:34:32.924 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T21:34:32.924 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T21:34:32.924 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T21:34:32.925 INFO:teuthology.provision.fog.smithi017:Deploy complete! 2024-02-10T21:34:32.926 DEBUG:teuthology.parallel:result is None 2024-02-10T21:34:33.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:34:36.412 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:34:42.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:34:45.468 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:34:51.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:34:54.560 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:35:00.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:35:03.644 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:35:09.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:35:12.732 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:35:18.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:35:21.792 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:35:27.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:35:30.876 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:35:36.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:35:39.936 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:35:45.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:35:49.020 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:35:55.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:35:58.108 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:36:04.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:36:07.168 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:36:13.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:36:16.252 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:36:22.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:36:25.312 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:36:31.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:36:34.396 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:36:40.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:36:43.484 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:36:49.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:36:52.540 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:36:58.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:37:01.596 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:37:07.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:37:10.652 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:37:16.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:37:19.708 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:37:25.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:37:28.768 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:37:34.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:37:37.852 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:37:43.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:37:46.908 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:37:52.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:37:55.968 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:38:01.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:38:05.052 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:38:11.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:38:14.140 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:38:20.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:38:23.200 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:38:29.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:38:32.284 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:38:38.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:38:41.340 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:38:47.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:38:50.428 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:38:56.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:38:59.484 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:39:05.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:39:08.544 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:39:14.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:39:17.628 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:39:23.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:39:26.720 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:39:32.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:39:35.772 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:39:41.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:39:44.860 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:39:50.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:39:53.916 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:39:59.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:40:02.972 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:40:08.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:40:12.032 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:40:18.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:40:21.116 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:40:27.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:40:30.172 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:40:36.179 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:40:39.260 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:40:45.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:40:48.316 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:40:54.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:40:57.372 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:41:03.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:41:06.464 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:41:12.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:41:15.548 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:41:21.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:41:24.604 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:41:30.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:41:33.696 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:41:39.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:41:42.780 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:41:48.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:41:51.868 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:41:57.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:42:00.956 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:42:06.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:42:10.044 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:42:16.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:42:19.100 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:42:25.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:42:28.156 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:42:34.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:42:37.212 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:42:43.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:42:46.268 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:42:52.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:42:55.324 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:43:01.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:43:04.412 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:43:10.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:43:13.468 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:43:19.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:43:22.556 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:43:28.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:43:31.644 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:43:37.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:43:40.700 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:43:46.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:43:49.788 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:43:55.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:43:58.876 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:44:04.883 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:44:07.968 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:44:13.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:44:17.052 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:44:23.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:44:26.108 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:44:32.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:44:35.164 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:44:41.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:44:44.252 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:44:50.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:44:53.344 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:44:59.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:45:02.428 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:45:08.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:45:11.484 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:45:17.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:45:20.540 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:45:26.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:45:29.628 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:45:35.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:45:38.684 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:45:44.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:45:47.776 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:45:53.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:45:56.860 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:46:02.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:46:05.916 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:46:11.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:46:14.972 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:46:20.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:46:24.032 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:46:30.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:46:33.084 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:46:39.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:46:42.140 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:46:48.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:46:51.196 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:46:57.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:47:00.252 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:47:06.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:47:09.340 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:47:15.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:47:18.396 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:47:24.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:47:27.452 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:47:33.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:47:36.512 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:47:42.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:47:45.596 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:47:51.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:47:54.684 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:48:00.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:48:03.740 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:48:09.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:48:12.828 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:48:18.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T21:48:21.884 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-10T21:48:21.886 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-10T21:48:21.888 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T21:48:22.192 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=20e58c9b3ef14e7f857e4e0548aa3275 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-10T21:48:22.195 INFO:teuthology.nuke:Checking targets against current locks 2024-02-10T21:48:22.263 DEBUG:teuthology.nuke:shortname: smithi017 2024-02-10T21:48:22.263 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T21:48:22.288 DEBUG:teuthology.nuke:shortname: smithi140 2024-02-10T21:48:22.288 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T21:48:22.292 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-10_17:31:55-rados-wip-yuri6-testing-2024-02-09-1422-distro-default-smithi/7555892', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 21:29:53.933956', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPg7eQRPm/3w4o5pAqyqeqIRH/qxncK7E45goNS1p0KeiDuM2NXOPGoy3ttPAc9oNTUSPl8iXfpEAeSbicLwABo='} 2024-02-10T21:48:22.316 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-10_17:31:55-rados-wip-yuri6-testing-2024-02-09-1422-distro-default-smithi/7555892', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 21:29:53.935452', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFu8FytB8vYJyJV2ZxgNJiZjKGFvRmFF0f71/vEQBogaBZyFD6yGGbSAmpon1nA6pff4OIulJi885+bXS8VSTs='} 2024-02-10T21:48:22.343 INFO:teuthology.orchestra.console.smithi017:Power off 2024-02-10T21:48:22.344 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T21:48:22.371 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off 2024-02-10T21:48:22.371 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:48:22.388 INFO:teuthology.orchestra.console.smithi140:Power off 2024-02-10T21:48:22.388 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T21:48:22.397 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-02-10T21:48:22.410 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2024-02-10T21:48:22.411 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:48:22.432 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-10T21:48:26.403 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:48:26.434 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:48:26.512 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-02-10T21:48:26.543 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-10T21:48:30.516 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:48:30.544 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:48:30.628 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-02-10T21:48:30.654 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-10T21:48:30.754 INFO:teuthology.orchestra.console.smithi140:Power off completed 2024-02-10T21:48:30.914 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2024-02-10T21:48:34.630 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T21:48:34.741 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-02-10T21:48:34.841 INFO:teuthology.orchestra.console.smithi017:Power off completed 2024-02-10T21:48:35.007 INFO:teuthology.lock.ops:unlocked: smithi017.front.sepia.ceph.com 2024-02-10T21:48:35.244 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds