2024-04-18T05:30:01.986 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-18T05:30:01.986 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:30:02.132 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f409a636c10>, signals=[15]) 2024-04-18T05:30:02.132 INFO:teuthology.lock.ops:Start node 'smithi100.front.sepia.ceph.com' reimaging 2024-04-18T05:30:02.132 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T05:30:02.132 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-04-18T05:30:02.151 INFO:teuthology.lock.ops:Start node 'smithi144.front.sepia.ceph.com' reimaging 2024-04-18T05:30:02.151 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T05:30:02.151 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-04-18T05:30:02.275 DEBUG:teuthology.provision.fog.smithi100:Requesting image smithi_centos_8.stream (ID 3) 2024-04-18T05:30:02.292 DEBUG:teuthology.provision.fog.smithi144:Requesting image smithi_centos_8.stream (ID 3) 2024-04-18T05:30:02.318 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of centos 8.stream 2024-04-18T05:30:02.342 INFO:teuthology.provision.fog.smithi144:Scheduling deploy of centos 8.stream 2024-04-18T05:30:02.395 DEBUG:teuthology.provision.fog.smithi144:Canceling deploy task with ID 969922 2024-04-18T05:30:02.564 INFO:teuthology.orchestra.console.smithi100:Power off 2024-04-18T05:30:02.565 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T05:30:02.586 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-04-18T05:30:02.587 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:30:02.609 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-04-18T05:30:02.627 INFO:teuthology.orchestra.console.smithi144:Power off 2024-04-18T05:30:02.627 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T05:30:02.649 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-04-18T05:30:02.649 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:30:02.670 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-04-18T05:30:02.710 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-04-18T05:30:02.771 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-04-18T05:30:02.811 INFO:teuthology.orchestra.console.smithi100:Power on 2024-04-18T05:30:02.811 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T05:30:02.833 DEBUG:teuthology.orchestra.console.smithi100:power on output: Chassis Power Control: Up/On 2024-04-18T05:30:02.833 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:30:02.854 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-04-18T05:30:02.872 INFO:teuthology.orchestra.console.smithi144:Power on 2024-04-18T05:30:02.873 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T05:30:02.894 DEBUG:teuthology.orchestra.console.smithi144:power on output: Chassis Power Control: Up/On 2024-04-18T05:30:02.895 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:30:02.917 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-04-18T05:30:06.858 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:30:06.918 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:30:06.969 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-04-18T05:30:07.028 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-18T05:30:07.069 INFO:teuthology.orchestra.console.smithi100:Power on completed 2024-04-18T05:30:07.129 INFO:teuthology.orchestra.console.smithi144:Power on completed 2024-04-18T05:30:07.171 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish 2024-04-18T05:30:07.230 INFO:teuthology.provision.fog.smithi144:Waiting for deploy to finish 2024-04-18T05:30:37.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:30:39.266 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:30:45.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:30:45.410 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:30:51.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:30:51.555 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:30:57.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:30:57.694 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:31:03.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:31:03.838 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:31:09.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:31:09.986 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:31:15.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:31:16.126 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:31:22.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:31:22.275 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:31:28.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:31:28.414 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:31:34.419 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:31:34.558 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:31:40.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:31:40.702 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:31:46.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:32:05.023 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:32:11.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:32:12.885 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:32:18.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:32:19.022 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-04-18T05:32:23.113 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-04-18T05:32:30.241 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-04-18T05:32:40.328 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-04-18T05:32:52.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:32:53.459 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-04-18T05:32:56.034 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-18T05:33:02.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:33:05.086 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-18T05:33:09.462 ERROR:paramiko.transport:Exception (client): Error reading SSH protocol banner 2024-04-18T05:33:09.463 ERROR:paramiko.transport:Traceback (most recent call last): 2024-04-18T05:33:09.464 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2327, in _check_banner 2024-04-18T05:33:09.464 ERROR:paramiko.transport: buf = self.packetizer.readline(timeout) 2024-04-18T05:33:09.464 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 381, in readline 2024-04-18T05:33:09.464 ERROR:paramiko.transport: buf += self._read_timeout(timeout) 2024-04-18T05:33:09.464 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 618, in _read_timeout 2024-04-18T05:33:09.464 ERROR:paramiko.transport: raise EOFError() 2024-04-18T05:33:09.464 ERROR:paramiko.transport:EOFError 2024-04-18T05:33:09.464 ERROR:paramiko.transport: 2024-04-18T05:33:09.464 ERROR:paramiko.transport:During handling of the above exception, another exception occurred: 2024-04-18T05:33:09.464 ERROR:paramiko.transport: 2024-04-18T05:33:09.464 ERROR:paramiko.transport:Traceback (most recent call last): 2024-04-18T05:33:09.464 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2143, in run 2024-04-18T05:33:09.464 ERROR:paramiko.transport: self._check_banner() 2024-04-18T05:33:09.464 ERROR:paramiko.transport: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2331, in _check_banner 2024-04-18T05:33:09.464 ERROR:paramiko.transport: raise SSHException( 2024-04-18T05:33:09.464 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2024-04-18T05:33:09.465 ERROR:paramiko.transport: 2024-04-18T05:33:09.465 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: SSHException('Error reading SSH protocol banner') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2327, in _check_banner buf = self.packetizer.readline(timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 381, in readline buf += self._read_timeout(timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 618, in _read_timeout raise EOFError() EOFError During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 451, in connect t.start_client(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 738, in start_client raise e File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2143, in run self._check_banner() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2331, in _check_banner raise SSHException( paramiko.ssh_exception.SSHException: Error reading SSH protocol banner 2024-04-18T05:33:11.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:33:14.143 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-18T05:33:20.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:33:23.198 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-18T05:33:29.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:33:32.254 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-18T05:33:38.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:33:41.310 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-18T05:33:47.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:33:50.366 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-04-18T05:33:56.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:34:03.038 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:34:09.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:34:12.098 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-04-18T05:34:18.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:34:49.894 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T05:34:49.895 DEBUG:teuthology.orchestra.run.smithi144:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T05:34:56.370 WARNING:teuthology.provision.fog.smithi100:timed out 2024-04-18T05:35:02.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:35:02.621 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T05:35:02.621 DEBUG:teuthology.orchestra.run.smithi100:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T05:35:02.876 INFO:teuthology.provision.fog.smithi100:Node is ready 2024-04-18T05:35:02.876 DEBUG:teuthology.orchestra.run.smithi100:> hostname 2024-04-18T05:35:02.896 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com 2024-04-18T05:35:02.896 DEBUG:teuthology.orchestra.run.smithi100:> grep smithi100.front.sepia.ceph.com /etc/hosts 2024-04-18T05:35:02.956 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100 2024-04-18T05:35:02.956 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2024-04-18T05:35:03.045 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi100.front.sepia.ceph.com/smithi100/g' /etc/hosts 2024-04-18T05:35:03.074 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/172.21.15.100/172.21.15.100/g' /etc/hosts 2024-04-18T05:35:03.143 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi100.front.sepia.ceph.com/smithi100/g' /etc/hostname 2024-04-18T05:35:03.211 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2024-04-18T05:35:03.278 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-04-18T05:35:03.336 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-04-18T05:35:03.336 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:35:03.337 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T05:35:03.338 INFO:teuthology.provision.fog.smithi100:Deploy complete! 2024-04-18T05:35:03.339 DEBUG:teuthology.parallel:result is None 2024-04-18T05:35:15.581 INFO:teuthology.provision.fog.smithi144:Node is ready 2024-04-18T05:35:15.581 DEBUG:teuthology.orchestra.run.smithi144:> hostname 2024-04-18T05:35:15.599 INFO:teuthology.orchestra.run.smithi144.stdout:smithi144 2024-04-18T05:35:15.599 DEBUG:teuthology.orchestra.run.smithi144:> grep smithi144 /etc/hosts 2024-04-18T05:35:15.657 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144 smithi144 smithi144 2024-04-18T05:35:15.657 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname smithi144 2024-04-18T05:35:15.725 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/smithi144/smithi144/g' /etc/hosts 2024-04-18T05:35:15.795 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/172.21.15.144/172.21.15.144/g' /etc/hosts 2024-04-18T05:35:15.863 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/smithi144/smithi144/g' /etc/hostname 2024-04-18T05:35:15.933 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname smithi144 2024-04-18T05:35:16.000 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-04-18T05:35:16.059 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-04-18T05:35:16.059 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="8" 2024-04-18T05:35:16.059 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-04-18T05:35:16.059 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-04-18T05:35:16.059 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="8" 2024-04-18T05:35:16.059 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:35:16.060 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-18T05:35:16.060 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-04-18T05:35:16.060 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-18T05:35:16.060 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-04-18T05:35:16.060 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:35:16.060 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:35:16.060 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T05:35:16.061 INFO:teuthology.provision.fog.smithi144:Deploy complete! 2024-04-18T05:35:16.062 DEBUG:teuthology.parallel:result is None 2024-04-18T05:35:16.221 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com 2024-04-18T05:35:16.221 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T05:35:16.239 INFO:teuthology.lock.ops:Checking smithi144.front.sepia.ceph.com 2024-04-18T05:35:16.241 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server 2024-04-18T05:35:16.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-18T05:35:16.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:35:16.379 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-04-18T05:35:16.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-04-18T05:35:16.517 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-04-18T05:35:16.576 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-04-18T05:35:16.576 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-04-18T05:35:16.636 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-04-18T05:35:16.636 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8" 2024-04-18T05:35:16.636 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-04-18T05:35:16.636 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-04-18T05:35:16.636 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8" 2024-04-18T05:35:16.637 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:35:16.637 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-18T05:35:16.637 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-04-18T05:35:16.637 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-18T05:35:16.637 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-04-18T05:35:16.637 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:35:16.638 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:35:16.638 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T05:35:16.639 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-04-18T05:35:16.657 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: set os type and version on server 2024-04-18T05:35:16.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-18T05:35:16.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:35:16.779 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-04-18T05:35:16.845 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-18T05:35:16.845 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-04-18T05:35:16.904 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-04-18T05:35:16.904 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-04-18T05:35:16.963 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-04-18T05:35:16.963 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="8" 2024-04-18T05:35:16.963 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-04-18T05:35:16.963 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-04-18T05:35:16.963 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="8" 2024-04-18T05:35:16.964 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:35:16.964 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-18T05:35:16.964 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-04-18T05:35:16.964 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-18T05:35:16.964 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-04-18T05:35:16.964 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:35:16.964 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:35:16.964 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T05:35:16.964 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-04-18T05:35:16.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:35:17.106 INFO:teuthology.dispatcher.supervisor:Running job 7661597 2024-04-18T05:35:17.106 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661597 --name teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/osd 2-workunit/suites/ffsb}} -- /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661597/orig.config.yaml 2024-04-18T05:35:17.118 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661597 2024-04-18T05:35:17.119 INFO:teuthology.dispatcher.supervisor:Job PID: 1652795 2024-04-18T05:35:17.119 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-18T05:37:17.121 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:39:17.199 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:41:17.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:43:17.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:45:17.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:47:17.469 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:49:17.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:51:17.693 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:53:17.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:53:17.871 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-04-18T05:53:17.990 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi144 but it was locked by another job: /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661602