2023-06-21T23:52:59.744 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-21T23:52:59.745 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-21T23:52:59.746 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-21T23:53:03.952 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8e4bb97430>, signals=[15]) 2023-06-21T23:53:03.953 INFO:teuthology.lock.ops:Start node 'smithi081.front.sepia.ceph.com' reimaging 2023-06-21T23:53:03.954 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: reset os type and version on server 2023-06-21T23:53:03.954 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2023-06-21T23:53:03.972 INFO:teuthology.lock.ops:Start node 'smithi120.front.sepia.ceph.com' reimaging 2023-06-21T23:53:03.973 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: reset os type and version on server 2023-06-21T23:53:03.973 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2023-06-21T23:53:03.974 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /host {"name": "smithi081"} 2023-06-21T23:53:13.020 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /host {"name": "smithi120"} 2023-06-21T23:53:28.064 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-06-21T23:53:36.527 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-06-21T23:53:41.535 DEBUG:teuthology.provision.fog.smithi081:Requesting image smithi_rhel_8.6 (ID 5) 2023-06-21T23:53:41.536 DEBUG:teuthology.provision.fog.smithi081:FOG request: PUT /host/186 {"imageID": 5} 2023-06-21T23:53:43.214 DEBUG:teuthology.provision.fog.smithi120:Requesting image smithi_rhel_8.6 (ID 5) 2023-06-21T23:53:43.215 DEBUG:teuthology.provision.fog.smithi120:FOG request: PUT /host/225 {"imageID": 5} 2023-06-21T23:53:43.219 INFO:teuthology.provision.fog.smithi081:Scheduling deploy of rhel 8.6 2023-06-21T23:53:43.219 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None 2023-06-21T23:53:43.276 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.6 2023-06-21T23:53:43.277 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-06-21T23:53:43.294 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /tasktype {"name": "deploy"} 2023-06-21T23:53:43.325 DEBUG:teuthology.provision.fog.smithi081:FOG request: POST /host/186/task {"taskTypeID": 1} 2023-06-21T23:53:48.334 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /tasktype {"name": "deploy"} 2023-06-21T23:53:53.381 DEBUG:teuthology.provision.fog.smithi120:FOG request: POST /host/225/task {"taskTypeID": 1} 2023-06-21T23:53:58.389 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None 2023-06-21T23:54:03.462 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-06-21T23:54:15.684 INFO:teuthology.orchestra.console:Power off smithi081 2023-06-21T23:54:15.685 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-21T23:54:15.712 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:18.577 INFO:teuthology.orchestra.console:Power off smithi120 2023-06-21T23:54:18.578 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-21T23:54:18.599 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:18.645 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2023-06-21T23:54:18.726 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2023-06-21T23:54:18.746 INFO:teuthology.orchestra.console:Power on smithi081 2023-06-21T23:54:18.747 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-21T23:54:18.770 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:18.828 INFO:teuthology.orchestra.console:Power on smithi120 2023-06-21T23:54:18.829 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-21T23:54:18.848 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:22.792 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:22.869 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:27.905 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:27.980 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:33.017 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:33.090 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:38.129 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:38.204 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:43.244 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:43.316 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:48.356 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:48.426 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:54:48.638 INFO:teuthology.orchestra.console:Power on for smithi120 completed 2023-06-21T23:54:48.739 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish 2023-06-21T23:54:48.740 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-06-21T23:54:53.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:54:58.817 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:55:01.872 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-06-21T23:55:03.929 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:55:07.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:55:09.040 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:55:11.988 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-06-21T23:55:14.152 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:55:17.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:55:22.997 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-21T23:55:23.243 INFO:teuthology.orchestra.console:Power on for smithi081 completed 2023-06-21T23:55:23.344 INFO:teuthology.provision.fog.smithi081:Waiting for deploy to finish 2023-06-21T23:55:23.344 DEBUG:teuthology.provision.fog.smithi081:FOG request: GET /task/active None 2023-06-21T23:55:25.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:55:30.853 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-06-21T23:55:32.596 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-06-21T23:55:36.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:55:38.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:55:46.672 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-06-21T23:55:52.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:55:57.684 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-06-21T23:56:00.754 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-06-21T23:56:03.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:56:08.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:56:13.765 ERROR:teuthology.orchestra.connection:Error authenticating with smithi120.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:13.774 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:24.848 ERROR:teuthology.orchestra.connection:Error authenticating with smithi120.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:24.856 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:29.321 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:29.354 ERROR:teuthology.orchestra.connection:Error authenticating with smithi120.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:35.328 ERROR:paramiko.transport:Exception (client): Error reading SSH protocol banner 2023-06-21T23:56:35.329 ERROR:paramiko.transport:Traceback (most recent call last): 2023-06-21T23:56:35.329 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2270, in _check_banner 2023-06-21T23:56:35.330 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2023-06-21T23:56:35.330 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 380, in readline 2023-06-21T23:56:35.330 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2023-06-21T23:56:35.330 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 609, in _read_timeout 2023-06-21T23:56:35.330 ERROR:paramiko.transport: raise EOFError() 2023-06-21T23:56:35.331 ERROR:paramiko.transport:EOFError 2023-06-21T23:56:35.331 ERROR:paramiko.transport: 2023-06-21T23:56:35.331 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2023-06-21T23:56:35.331 ERROR:paramiko.transport: 2023-06-21T23:56:35.331 ERROR:paramiko.transport:Traceback (most recent call last): 2023-06-21T23:56:35.332 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2093, in run 2023-06-21T23:56:35.332 ERROR:paramiko.transport: self._check_banner() 2023-06-21T23:56:35.332 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2274, in _check_banner 2023-06-21T23:56:35.332 ERROR:paramiko.transport: raise SSHException( 2023-06-21T23:56:35.333 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2023-06-21T23:56:35.333 ERROR:paramiko.transport: 2023-06-21T23:56:35.333 WARNING:teuthology.provision.fog.smithi081:Error reading SSH protocol banner 2023-06-21T23:56:35.441 ERROR:teuthology.orchestra.connection:Error authenticating with smithi120.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:41.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:56:46.522 ERROR:teuthology.orchestra.connection:Error authenticating with smithi120.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:46.565 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:52.631 ERROR:teuthology.orchestra.connection:Error authenticating with smithi120.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:52.646 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:58.709 ERROR:teuthology.orchestra.connection:Error authenticating with smithi120.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:56:58.726 ERROR:teuthology.orchestra.connection:Error authenticating with smithi081.front.sepia.ceph.com: Authentication failed. 2023-06-21T23:58:04.713 WARNING:teuthology.provision.fog.smithi120:timed out 2023-06-21T23:58:09.717 WARNING:teuthology.provision.fog.smithi081:timed out 2023-06-21T23:58:10.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:58:15.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:58:26.836 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-06-21T23:58:32.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:58:37.844 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-06-21T23:58:40.912 WARNING:teuthology.provision.fog.smithi081:[Errno None] Unable to connect to port 22 on 172.21.15.81 2023-06-21T23:58:43.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:58:48.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-21T23:58:53.927 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-21T23:58:53.927 DEBUG:teuthology.orchestra.run.smithi081:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-21T23:58:53.996 INFO:teuthology.provision.fog.smithi081:Node is ready 2023-06-21T23:58:53.997 DEBUG:teuthology.orchestra.run.smithi081:> hostname 2023-06-21T23:58:54.054 INFO:teuthology.orchestra.run.smithi081.stdout:smithi081 2023-06-21T23:58:54.055 DEBUG:teuthology.orchestra.run.smithi081:> grep smithi081 /etc/hosts 2023-06-21T23:58:54.113 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81 smithi081 smithi081 2023-06-21T23:58:54.114 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname smithi081 2023-06-21T23:58:54.159 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-21T23:58:54.159 DEBUG:teuthology.orchestra.run.smithi120:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-21T23:58:54.183 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi081/smithi081/g' /etc/hosts 2023-06-21T23:58:54.252 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/172.21.15.81/172.21.15.81/g' /etc/hosts 2023-06-21T23:58:54.320 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi081/smithi081/g' /etc/hostname 2023-06-21T23:58:54.388 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname smithi081 2023-06-21T23:58:54.428 INFO:teuthology.provision.fog.smithi120:Node is ready 2023-06-21T23:58:54.428 DEBUG:teuthology.orchestra.run.smithi120:> hostname 2023-06-21T23:58:54.445 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120.front.sepia.ceph.com 2023-06-21T23:58:54.446 DEBUG:teuthology.orchestra.run.smithi120:> grep smithi120.front.sepia.ceph.com /etc/hosts 2023-06-21T23:58:54.454 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2023-06-21T23:58:54.503 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120 2023-06-21T23:58:54.503 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname smithi120 2023-06-21T23:58:54.511 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="Red Hat Enterprise Linux" 2023-06-21T23:58:54.512 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="8.6 (Ootpa)" 2023-06-21T23:58:54.512 INFO:teuthology.orchestra.run.smithi081.stdout:ID="rhel" 2023-06-21T23:58:54.512 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="fedora" 2023-06-21T23:58:54.512 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="8.6" 2023-06-21T23:58:54.512 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el8" 2023-06-21T23:58:54.512 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-21T23:58:54.513 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2023-06-21T23:58:54.513 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-21T23:58:54.513 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.redhat.com/" 2023-06-21T23:58:54.513 INFO:teuthology.orchestra.run.smithi081.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-21T23:58:54.513 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-21T23:58:54.513 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-06-21T23:58:54.513 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-21T23:58:54.513 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-21T23:58:54.514 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-21T23:58:54.514 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-21T23:58:54.514 INFO:teuthology.provision.fog.smithi081:Deploy complete! 2023-06-21T23:58:54.516 DEBUG:teuthology.parallel:result is None 2023-06-21T23:58:54.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/smithi120.front.sepia.ceph.com/smithi120/g' /etc/hosts 2023-06-21T23:58:54.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/172.21.15.120/172.21.15.120/g' /etc/hosts 2023-06-21T23:58:54.704 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/smithi120.front.sepia.ceph.com/smithi120/g' /etc/hostname 2023-06-21T23:58:54.770 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname smithi120 2023-06-21T23:58:54.836 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2023-06-21T23:58:54.893 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux" 2023-06-21T23:58:54.893 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.6 (Ootpa)" 2023-06-21T23:58:54.893 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel" 2023-06-21T23:58:54.894 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora" 2023-06-21T23:58:54.894 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.6" 2023-06-21T23:58:54.894 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8" 2023-06-21T23:58:54.894 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-21T23:58:54.894 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2023-06-21T23:58:54.894 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-21T23:58:54.895 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/" 2023-06-21T23:58:54.895 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-21T23:58:54.895 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-21T23:58:54.895 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-06-21T23:58:54.895 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-21T23:58:54.895 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-21T23:58:54.895 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-21T23:58:54.895 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-21T23:58:54.896 INFO:teuthology.provision.fog.smithi120:Deploy complete! 2023-06-21T23:58:54.897 DEBUG:teuthology.parallel:result is None 2023-06-21T23:59:59.075 ERROR:teuthology.misc:write (smithi081.front.sepia.ceph.com): No route to host 2023-06-21T23:59:59.076 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 326, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 290, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/misc.py", line 1106, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi081.front.sepia.ceph.com' reached maximum tries (6) after waiting for 5 seconds 2023-06-21T23:59:59.077 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-21T23:59:59.078 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T00:00:02.700 INFO:teuthology.nuke:Checking targets against current locks 2023-06-22T00:00:07.722 DEBUG:teuthology.nuke:shortname: smithi081 2023-06-22T00:00:07.723 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T00:00:07.746 DEBUG:teuthology.nuke:shortname: smithi120 2023-06-22T00:00:07.746 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T00:00:07.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-21_18:12:20-rados-wip-yuri5-testing-2023-06-21-0750-distro-default-smithi/7310413', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-21 23:59:43.071784', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE8FWMgD95CwBN7eJQkgW+IXb74CI8Hz4zsm3rEQXuoBq0c3+WFy4Rh7J05YKPQeyNLNtsNKIQkCp/G1kZY/dR4='} 2023-06-22T00:00:07.771 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-21_18:12:20-rados-wip-yuri5-testing-2023-06-21-0750-distro-default-smithi/7310397', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-21 23:52:43.920233', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMkM83wYw+Am7GWKnbKbBzApc56Iqkblx31nSkAeWyqBeiPy9hOWmw0r6Dn0u/IlE1px8Id38CGqvSzmO9dbmPg='} 2023-06-22T00:00:07.805 INFO:teuthology.orchestra.console:Power off smithi081 2023-06-22T00:00:07.806 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T00:00:07.829 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T00:00:07.849 INFO:teuthology.orchestra.console:Power off smithi120 2023-06-22T00:00:07.850 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T00:00:07.873 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T00:00:07.962 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2023-06-22T00:00:08.122 INFO:teuthology.lock.ops:unlocked: smithi081.front.sepia.ceph.com 2023-06-22T00:00:11.898 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T00:00:17.012 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T00:00:22.125 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T00:00:27.236 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T00:00:27.449 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2023-06-22T00:00:28.575 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com 2023-06-22T00:00:33.868 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 326, in reimage_machines reimaged = do_update_keys(list(reimaged.keys()))[1] File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 290, in do_update_keys keys_dict = misc.ssh_keyscan(machines, _raise=_raise) File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/misc.py", line 1106, in ssh_keyscan while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: 'ssh_keyscan smithi081.front.sepia.ceph.com' reached maximum tries (6) after waiting for 5 seconds