2024-02-19T07:09:12.474 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-19T07:09:12.474 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:09:12.650 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0da1790c10>, signals=[15]) 2024-02-19T07:09:12.651 INFO:teuthology.lock.ops:Start node 'smithi008.front.sepia.ceph.com' reimaging 2024-02-19T07:09:12.651 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: reset os type and version on server 2024-02-19T07:09:12.651 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2024-02-19T07:09:12.675 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging 2024-02-19T07:09:12.675 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server 2024-02-19T07:09:12.675 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-02-19T07:09:12.691 INFO:teuthology.lock.ops:Start node 'smithi173.front.sepia.ceph.com' reimaging 2024-02-19T07:09:12.691 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: reset os type and version on server 2024-02-19T07:09:12.691 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-02-19T07:09:12.792 DEBUG:teuthology.provision.fog.smithi008:Requesting image smithi_centos_9.stream (ID 10) 2024-02-19T07:09:12.813 DEBUG:teuthology.provision.fog.smithi167:Requesting image smithi_centos_9.stream (ID 10) 2024-02-19T07:09:12.821 DEBUG:teuthology.provision.fog.smithi173:Requesting image smithi_centos_9.stream (ID 10) 2024-02-19T07:09:12.836 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of centos 9.stream 2024-02-19T07:09:12.861 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 9.stream 2024-02-19T07:09:12.870 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of centos 9.stream 2024-02-19T07:09:13.108 INFO:teuthology.orchestra.console.smithi173:Power off 2024-02-19T07:09:13.108 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-19T07:09:13.128 INFO:teuthology.orchestra.console.smithi167:Power off 2024-02-19T07:09:13.128 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-19T07:09:13.144 INFO:teuthology.orchestra.console.smithi008:Power off 2024-02-19T07:09:13.144 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-19T07:09:13.154 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2024-02-19T07:09:13.154 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:13.163 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2024-02-19T07:09:13.164 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:13.174 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2024-02-19T07:09:13.174 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:13.183 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-02-19T07:09:13.187 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-02-19T07:09:13.197 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-02-19T07:09:17.188 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:17.195 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:17.203 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:17.297 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-02-19T07:09:17.304 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-02-19T07:09:17.313 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-02-19T07:09:21.300 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:21.307 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:21.315 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:21.409 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2024-02-19T07:09:21.416 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-02-19T07:09:21.424 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-02-19T07:09:21.510 INFO:teuthology.orchestra.console.smithi167:Power off completed 2024-02-19T07:09:21.517 INFO:teuthology.orchestra.console.smithi173:Power off completed 2024-02-19T07:09:21.524 INFO:teuthology.orchestra.console.smithi008:Power off completed 2024-02-19T07:09:21.612 INFO:teuthology.orchestra.console.smithi167:Power on 2024-02-19T07:09:21.612 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-19T07:09:21.620 INFO:teuthology.orchestra.console.smithi173:Power on 2024-02-19T07:09:21.621 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-19T07:09:21.627 INFO:teuthology.orchestra.console.smithi008:Power on 2024-02-19T07:09:21.627 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-19T07:09:21.637 DEBUG:teuthology.orchestra.console.smithi167:power on output: Chassis Power Control: Up/On 2024-02-19T07:09:21.638 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:21.646 DEBUG:teuthology.orchestra.console.smithi173:power on output: Chassis Power Control: Up/On 2024-02-19T07:09:21.646 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:21.653 DEBUG:teuthology.orchestra.console.smithi008:power on output: Chassis Power Control: Up/On 2024-02-19T07:09:21.654 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:21.663 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2024-02-19T07:09:21.666 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-02-19T07:09:21.676 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-02-19T07:09:25.664 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:25.670 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:25.677 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:25.773 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-02-19T07:09:25.779 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-02-19T07:09:25.784 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-02-19T07:09:25.874 INFO:teuthology.orchestra.console.smithi167:Power on completed 2024-02-19T07:09:25.885 INFO:teuthology.orchestra.console.smithi008:Power on completed 2024-02-19T07:09:25.975 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish 2024-02-19T07:09:25.987 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish 2024-02-19T07:09:29.782 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T07:09:29.918 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-02-19T07:09:30.019 INFO:teuthology.orchestra.console.smithi173:Power on completed 2024-02-19T07:09:30.119 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2024-02-19T07:11:56.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:11:59.964 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-19T07:12:00.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:03.964 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-02-19T07:12:05.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:09.020 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-19T07:12:09.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:11.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:13.024 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-02-19T07:12:15.004 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-02-19T07:12:15.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:18.076 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-19T07:12:19.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:21.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:22.108 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-02-19T07:12:24.064 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-02-19T07:12:24.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:27.132 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-19T07:12:28.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:30.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:31.164 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-02-19T07:12:33.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:33.152 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-02-19T07:12:36.191 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-19T07:12:37.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:39.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:40.220 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-02-19T07:12:42.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:42.236 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-02-19T07:12:45.244 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-02-19T07:12:46.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:48.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:49.280 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-02-19T07:12:51.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:51.324 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-02-19T07:12:55.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:57.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:12:58.368 WARNING:teuthology.provision.fog.smithi173:[Errno None] Unable to connect to port 22 on 172.21.15.173 2024-02-19T07:13:00.380 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-02-19T07:13:04.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:13:06.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:13:09.436 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-02-19T07:13:15.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:13:18.496 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-02-19T07:13:24.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:13:51.276 WARNING:teuthology.provision.fog.smithi008:timed out 2024-02-19T07:13:57.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:13:57.573 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-19T07:13:57.574 DEBUG:teuthology.orchestra.run.smithi008:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-19T07:13:57.909 INFO:teuthology.provision.fog.smithi008:Node is ready 2024-02-19T07:13:57.909 DEBUG:teuthology.orchestra.run.smithi008:> hostname 2024-02-19T07:13:57.924 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com 2024-02-19T07:13:57.924 DEBUG:teuthology.orchestra.run.smithi008:> grep smithi008.front.sepia.ceph.com /etc/hosts 2024-02-19T07:13:57.980 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008 2024-02-19T07:13:57.980 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname smithi008 2024-02-19T07:13:58.094 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/smithi008.front.sepia.ceph.com/smithi008/g' /etc/hosts 2024-02-19T07:13:58.126 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/172.21.15.8/172.21.15.8/g' /etc/hosts 2024-02-19T07:13:58.197 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/smithi008.front.sepia.ceph.com/smithi008/g' /etc/hostname 2024-02-19T07:13:58.269 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname smithi008 2024-02-19T07:13:58.339 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="9" 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="9" 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el9" 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2024-02-19T07:13:58.394 INFO:teuthology.orchestra.run.smithi008.stdout:LOGO="fedora-logo-icon" 2024-02-19T07:13:58.395 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T07:13:58.395 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2024-02-19T07:13:58.395 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T07:13:58.395 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T07:13:58.395 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T07:13:58.395 INFO:teuthology.provision.fog.smithi008:Deploy complete! 2024-02-19T07:13:58.397 DEBUG:teuthology.parallel:result is None 2024-02-19T07:14:04.380 WARNING:teuthology.provision.fog.smithi173:timed out 2024-02-19T07:14:10.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:14:10.633 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-19T07:14:10.634 DEBUG:teuthology.orchestra.run.smithi173:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-19T07:14:10.947 INFO:teuthology.provision.fog.smithi173:Node is ready 2024-02-19T07:14:10.947 DEBUG:teuthology.orchestra.run.smithi173:> hostname 2024-02-19T07:14:10.961 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com 2024-02-19T07:14:10.962 DEBUG:teuthology.orchestra.run.smithi173:> grep smithi173.front.sepia.ceph.com /etc/hosts 2024-02-19T07:14:11.017 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173 2024-02-19T07:14:11.018 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2024-02-19T07:14:11.132 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hosts 2024-02-19T07:14:11.160 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/172.21.15.173/172.21.15.173/g' /etc/hosts 2024-02-19T07:14:11.227 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hostname 2024-02-19T07:14:11.294 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2024-02-19T07:14:11.360 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-02-19T07:14:11.414 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T07:14:11.415 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T07:14:11.416 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2024-02-19T07:14:11.416 DEBUG:teuthology.parallel:result is None 2024-02-19T07:14:24.516 WARNING:teuthology.provision.fog.smithi167:timed out 2024-02-19T07:14:30.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:14:31.005 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-19T07:14:31.005 DEBUG:teuthology.orchestra.run.smithi167:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-19T07:14:31.322 INFO:teuthology.provision.fog.smithi167:Node is ready 2024-02-19T07:14:31.323 DEBUG:teuthology.orchestra.run.smithi167:> hostname 2024-02-19T07:14:31.338 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com 2024-02-19T07:14:31.339 DEBUG:teuthology.orchestra.run.smithi167:> grep smithi167.front.sepia.ceph.com /etc/hosts 2024-02-19T07:14:31.393 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167 2024-02-19T07:14:31.394 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167 2024-02-19T07:14:31.513 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hosts 2024-02-19T07:14:31.542 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/172.21.15.167/172.21.15.167/g' /etc/hosts 2024-02-19T07:14:31.610 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hostname 2024-02-19T07:14:31.677 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167 2024-02-19T07:14:31.745 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-02-19T07:14:31.799 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-02-19T07:14:31.799 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T07:14:31.800 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T07:14:31.801 INFO:teuthology.provision.fog.smithi167:Deploy complete! 2024-02-19T07:14:31.801 DEBUG:teuthology.parallel:result is None 2024-02-19T07:14:32.019 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com 2024-02-19T07:14:32.019 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-19T07:14:32.040 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com 2024-02-19T07:14:32.040 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-19T07:14:32.065 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2024-02-19T07:14:32.065 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-19T07:14:32.082 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server 2024-02-19T07:14:32.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-02-19T07:14:32.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:14:32.191 DEBUG:teuthology.orchestra.run.smithi008:> true 2024-02-19T07:14:32.374 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-02-19T07:14:32.375 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2024-02-19T07:14:32.430 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2024-02-19T07:14:32.430 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2024-02-19T07:14:32.486 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2024-02-19T07:14:32.486 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="9" 2024-02-19T07:14:32.486 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2024-02-19T07:14:32.486 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="9" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el9" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:LOGO="fedora-logo-icon" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T07:14:32.487 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T07:14:32.488 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2024-02-19T07:14:32.507 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server 2024-02-19T07:14:32.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-02-19T07:14:32.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:14:32.593 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-02-19T07:14:32.658 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-02-19T07:14:32.658 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-02-19T07:14:32.716 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-02-19T07:14:32.716 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-02-19T07:14:32.775 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-02-19T07:14:32.775 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-02-19T07:14:32.775 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-02-19T07:14:32.775 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-02-19T07:14:32.775 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T07:14:32.776 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T07:14:32.777 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-02-19T07:14:32.799 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2024-02-19T07:14:32.799 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-02-19T07:14:32.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T07:14:32.888 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-02-19T07:14:33.058 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-02-19T07:14:33.058 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-02-19T07:14:33.113 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-02-19T07:14:33.113 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-02-19T07:14:33.167 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T07:14:33.168 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T07:14:33.169 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-02-19T07:14:33.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:14:33.291 INFO:teuthology.dispatcher.supervisor:Running job 7566181 2024-02-19T07:14:33.292 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-19_06:18:55-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7566181 --name vshankar-2024-02-19_06:18:55-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/postgres}} -- /home/teuthworker/archive/vshankar-2024-02-19_06:18:55-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7566181/orig.config.yaml 2024-02-19T07:14:33.295 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-19_06:18:55-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7566181 2024-02-19T07:14:33.295 INFO:teuthology.dispatcher.supervisor:Job PID: 1796390 2024-02-19T07:14:33.295 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-19T07:16:33.390 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:18:33.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:20:33.717 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:22:33.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:24:34.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:26:34.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:28:34.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:30:34.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T07:30:34.573 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1