2024-02-16T06:01:45.048 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-16T06:01:45.048 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T06:01:45.193 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcc2c073c10>, signals=[15]) 2024-02-16T06:01:45.194 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging 2024-02-16T06:01:45.194 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server 2024-02-16T06:01:45.194 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-02-16T06:01:45.213 INFO:teuthology.lock.ops:Start node 'smithi140.front.sepia.ceph.com' reimaging 2024-02-16T06:01:45.214 INFO:teuthology.lock.ops:Updating [smithi140.front.sepia.ceph.com]: reset os type and version on server 2024-02-16T06:01:45.214 INFO:teuthology.lock.ops:Updating smithi140.front.sepia.ceph.com on lock server 2024-02-16T06:01:45.332 DEBUG:teuthology.provision.fog.smithi050:Requesting image smithi_centos_9.stream (ID 10) 2024-02-16T06:01:45.353 DEBUG:teuthology.provision.fog.smithi140:Requesting image smithi_centos_9.stream (ID 10) 2024-02-16T06:01:45.394 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of centos 9.stream 2024-02-16T06:01:45.410 INFO:teuthology.provision.fog.smithi140:Scheduling deploy of centos 9.stream 2024-02-16T06:01:45.669 INFO:teuthology.orchestra.console.smithi140:Power off 2024-02-16T06:01:45.670 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T06:01:45.685 INFO:teuthology.orchestra.console.smithi050:Power off 2024-02-16T06:01:45.685 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T06:01:45.694 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2024-02-16T06:01:45.695 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:45.707 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2024-02-16T06:01:45.708 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:45.719 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-16T06:01:45.733 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-16T06:01:45.821 INFO:teuthology.orchestra.console.smithi140:Power off completed 2024-02-16T06:01:45.923 INFO:teuthology.orchestra.console.smithi140:Power on 2024-02-16T06:01:45.923 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-16T06:01:45.946 DEBUG:teuthology.orchestra.console.smithi140:power on output: Chassis Power Control: Up/On 2024-02-16T06:01:45.946 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:45.968 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-16T06:01:49.736 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:49.845 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-16T06:01:49.970 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:50.079 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-16T06:01:53.849 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:53.958 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2024-02-16T06:01:54.059 INFO:teuthology.orchestra.console.smithi050:Power off completed 2024-02-16T06:01:54.080 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:54.161 INFO:teuthology.orchestra.console.smithi050:Power on 2024-02-16T06:01:54.161 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-16T06:01:54.186 DEBUG:teuthology.orchestra.console.smithi050:power on output: Chassis Power Control: Up/On 2024-02-16T06:01:54.186 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:54.195 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-16T06:01:54.209 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2024-02-16T06:01:54.296 INFO:teuthology.orchestra.console.smithi140:Power on completed 2024-02-16T06:01:54.396 INFO:teuthology.provision.fog.smithi140:Waiting for deploy to finish 2024-02-16T06:01:58.212 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:01:58.321 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-16T06:01:58.421 INFO:teuthology.orchestra.console.smithi050:Power on completed 2024-02-16T06:01:58.522 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2024-02-16T06:04:29.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:04:32.576 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-02-16T06:04:38.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:04:40.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:04:41.660 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-02-16T06:04:43.612 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:04:47.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:04:49.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:04:50.716 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-02-16T06:04:52.668 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:04:56.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:04:58.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:04:59.776 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-02-16T06:05:01.724 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:05:05.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:05:07.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:05:08.860 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-02-16T06:05:10.780 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:05:14.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:05:16.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:05:19.836 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:05:25.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:05:28.924 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:05:34.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:05:37.984 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:05:43.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:06:14.898 WARNING:teuthology.provision.fog.smithi050:timed out 2024-02-16T06:06:20.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:06:21.163 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-16T06:06:21.163 DEBUG:teuthology.orchestra.run.smithi050:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-16T06:06:21.493 INFO:teuthology.provision.fog.smithi050:Node is ready 2024-02-16T06:06:21.493 DEBUG:teuthology.orchestra.run.smithi050:> hostname 2024-02-16T06:06:21.509 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2024-02-16T06:06:21.509 DEBUG:teuthology.orchestra.run.smithi050:> grep smithi050.front.sepia.ceph.com /etc/hosts 2024-02-16T06:06:21.565 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2024-02-16T06:06:21.566 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2024-02-16T06:06:21.681 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hosts 2024-02-16T06:06:21.713 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/172.21.15.50/172.21.15.50/g' /etc/hosts 2024-02-16T06:06:21.785 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hostname 2024-02-16T06:06:21.856 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2024-02-16T06:06:21.927 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-02-16T06:06:21.981 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-02-16T06:06:21.981 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-02-16T06:06:21.981 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-02-16T06:06:21.981 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T06:06:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T06:06:21.983 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2024-02-16T06:06:21.985 DEBUG:teuthology.parallel:result is None 2024-02-16T06:06:44.012 WARNING:teuthology.provision.fog.smithi140:timed out 2024-02-16T06:06:50.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:07:08.348 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:07:14.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:07:17.408 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:07:23.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:07:26.464 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:07:32.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:07:35.548 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:07:41.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:07:44.636 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:07:50.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:07:53.724 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:07:59.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:08:02.780 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:08:08.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:08:11.868 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:08:17.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:08:20.956 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:08:26.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:08:30.012 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:08:36.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:08:39.072 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:08:45.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:08:48.160 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:08:54.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:08:57.244 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:09:03.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:09:06.300 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:09:12.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:09:15.356 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:09:21.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:09:24.416 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:09:30.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:09:33.500 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:09:39.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:09:42.556 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:09:48.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:09:51.612 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:09:57.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:10:00.704 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:10:06.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:10:09.788 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:10:15.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:10:18.848 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:10:24.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:10:27.932 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:10:33.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:10:36.988 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:10:42.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:10:46.044 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:10:52.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:10:55.136 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:11:01.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:11:04.220 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:11:10.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:11:13.280 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:11:19.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:11:22.364 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:11:28.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:11:31.452 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:11:37.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:11:40.508 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:11:46.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:11:49.568 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:11:55.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:11:58.652 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:12:04.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:12:07.740 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:12:13.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:12:16.831 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:12:22.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:12:25.920 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:12:31.925 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:12:35.004 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:12:41.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:12:44.064 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:12:50.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:12:53.148 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:12:59.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:13:02.204 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:13:08.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:13:11.295 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:13:17.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:13:20.380 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:13:26.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:13:29.436 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:13:35.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:13:38.492 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:13:44.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:13:47.548 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:13:53.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:13:56.608 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:14:02.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:14:05.692 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:14:11.699 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:14:14.784 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:14:20.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:14:23.868 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:14:29.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:14:32.924 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:14:38.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:14:42.012 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:14:48.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:14:51.068 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:14:57.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:15:00.124 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:15:06.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:15:09.212 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:15:15.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:15:18.304 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:15:24.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:15:27.388 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:15:33.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:15:36.444 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:15:42.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:15:45.500 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:15:51.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:15:54.588 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:16:00.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:16:03.680 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:16:09.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:16:12.764 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:16:18.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:16:21.852 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:16:27.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:16:30.908 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:16:36.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:16:39.968 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:16:45.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:16:49.052 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:16:55.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:16:58.108 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:17:04.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:17:07.196 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:17:13.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:17:16.252 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:17:22.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:17:25.312 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:17:31.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:17:34.396 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:17:40.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:17:43.484 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:17:49.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:17:52.540 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:17:58.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:18:01.596 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:18:07.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:18:10.684 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:18:16.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:18:19.740 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:18:25.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:18:28.796 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:18:34.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:18:37.856 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:18:43.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:18:46.944 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:18:52.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:18:56.028 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:19:02.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:19:05.120 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:19:11.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:19:14.208 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:19:20.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:19:23.292 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:19:29.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:19:32.540 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:19:38.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:19:41.596 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:19:47.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:19:50.652 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:19:56.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:19:59.712 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:20:05.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:20:08.796 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:20:14.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:20:17.884 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:20:23.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:20:26.944 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:20:32.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:20:36.028 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:20:42.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:20:45.084 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:20:51.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi140.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T06:20:54.140 WARNING:teuthology.provision.fog.smithi140:[Errno None] Unable to connect to port 22 on 172.21.15.140 2024-02-16T06:20:54.142 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-16T06:20:54.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T06:20:54.466 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=88db55ce7a51428e86120fcf8fe5baec Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds 2024-02-16T06:20:54.468 INFO:teuthology.nuke:Checking targets against current locks 2024-02-16T06:20:54.544 DEBUG:teuthology.nuke:shortname: smithi050 2024-02-16T06:20:54.544 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T06:20:54.565 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-15_15:09:46-rados-wip-yuri5-testing-2024-02-14-1335-distro-default-smithi/7561194', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 06:01:44.034301', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBPFgxugFYV2LYd60NnElnA0vMXiy92Shvul2vepv70DUzGw6vSqoPn5ZrKQqM//ha/21ryN4mZB/5kN9a9GKmc='} 2024-02-16T06:20:54.569 DEBUG:teuthology.nuke:shortname: smithi140 2024-02-16T06:20:54.569 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-16T06:20:54.594 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi140.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-15_15:09:46-rados-wip-yuri5-testing-2024-02-14-1335-distro-default-smithi/7561194', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-16 06:01:44.032842', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxXAJHvdwr3emMFb4ph9kLOsWTfJdJY2lZxCR3jrvoVuP3zFLiJmCS0eLoZWBUyDoabgyjn+LEX4KNzOFOjJQc='} 2024-02-16T06:20:54.622 INFO:teuthology.orchestra.console.smithi050:Power off 2024-02-16T06:20:54.622 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T06:20:54.652 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2024-02-16T06:20:54.652 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:20:54.676 INFO:teuthology.orchestra.console.smithi140:Power off 2024-02-16T06:20:54.676 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T06:20:54.689 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-16T06:20:54.709 DEBUG:teuthology.orchestra.console.smithi140:power off output: Chassis Power Control: Down/Off 2024-02-16T06:20:54.709 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:20:54.732 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-16T06:20:58.692 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:20:58.734 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:20:58.806 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-16T06:20:58.843 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-16T06:21:02.812 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:21:02.845 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:21:02.923 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-16T06:21:02.954 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is on 2024-02-16T06:21:06.928 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:21:06.955 DEBUG:teuthology.orchestra.console.smithi140:pexpect command: ipmitool -H smithi140.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T06:21:07.038 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2024-02-16T06:21:07.066 DEBUG:teuthology.orchestra.console.smithi140:check power output: Chassis Power is off 2024-02-16T06:21:07.140 INFO:teuthology.orchestra.console.smithi050:Power off completed 2024-02-16T06:21:07.168 INFO:teuthology.orchestra.console.smithi140:Power off completed 2024-02-16T06:21:07.292 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com 2024-02-16T06:21:07.316 INFO:teuthology.lock.ops:unlocked: smithi140.front.sepia.ceph.com 2024-02-16T06:21:07.559 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/dispatcher/supervisor.py", line 232, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/provision/fog.py", line 275, in _wait_for_ready while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (101) after waiting for 600 seconds