2024-03-21T08:39:23.230 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T08:39:23.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:39:23.397 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd4c77d3c10>, signals=[15]) 2024-03-21T08:39:23.398 INFO:teuthology.lock.ops:Start node 'smithi063.front.sepia.ceph.com' reimaging 2024-03-21T08:39:23.398 INFO:teuthology.lock.ops:Updating [smithi063.front.sepia.ceph.com]: reset os type and version on server 2024-03-21T08:39:23.398 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-03-21T08:39:23.421 INFO:teuthology.lock.ops:Start node 'smithi136.front.sepia.ceph.com' reimaging 2024-03-21T08:39:23.422 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: reset os type and version on server 2024-03-21T08:39:23.422 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-03-21T08:39:23.445 INFO:teuthology.lock.ops:Start node 'smithi137.front.sepia.ceph.com' reimaging 2024-03-21T08:39:23.445 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: reset os type and version on server 2024-03-21T08:39:23.445 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-03-21T08:39:23.545 DEBUG:teuthology.provision.fog.smithi063:Requesting image smithi_centos_9.stream (ID 10) 2024-03-21T08:39:23.579 DEBUG:teuthology.provision.fog.smithi136:Requesting image smithi_centos_9.stream (ID 10) 2024-03-21T08:39:23.587 INFO:teuthology.provision.fog.smithi063:Scheduling deploy of centos 9.stream 2024-03-21T08:39:23.590 DEBUG:teuthology.provision.fog.smithi137:Requesting image smithi_centos_9.stream (ID 10) 2024-03-21T08:39:23.626 INFO:teuthology.provision.fog.smithi136:Scheduling deploy of centos 9.stream 2024-03-21T08:39:23.653 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of centos 9.stream 2024-03-21T08:39:23.879 INFO:teuthology.orchestra.console.smithi063:Power off 2024-03-21T08:39:23.880 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T08:39:23.896 INFO:teuthology.orchestra.console.smithi136:Power off 2024-03-21T08:39:23.897 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T08:39:23.909 DEBUG:teuthology.orchestra.console.smithi063:power off output: Chassis Power Control: Down/Off 2024-03-21T08:39:23.909 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:23.919 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2024-03-21T08:39:23.919 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:23.935 INFO:teuthology.orchestra.console.smithi137:Power off 2024-03-21T08:39:23.935 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T08:39:23.944 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-21T08:39:23.944 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-03-21T08:39:23.957 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-03-21T08:39:23.957 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:23.982 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-21T08:39:27.948 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:27.958 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:27.982 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:28.061 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-21T08:39:28.068 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-03-21T08:39:28.091 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-21T08:39:32.066 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:32.074 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:32.092 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:32.177 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-21T08:39:32.183 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off 2024-03-21T08:39:32.201 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-21T08:39:32.284 INFO:teuthology.orchestra.console.smithi063:Power off completed 2024-03-21T08:39:32.385 INFO:teuthology.orchestra.console.smithi063:Power on 2024-03-21T08:39:32.385 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-21T08:39:32.407 DEBUG:teuthology.orchestra.console.smithi063:power on output: Chassis Power Control: Up/On 2024-03-21T08:39:32.408 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:32.429 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off 2024-03-21T08:39:36.182 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:36.201 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:36.292 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-03-21T08:39:36.310 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-03-21T08:39:36.393 INFO:teuthology.orchestra.console.smithi136:Power off completed 2024-03-21T08:39:36.411 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-03-21T08:39:36.430 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:36.494 INFO:teuthology.orchestra.console.smithi136:Power on 2024-03-21T08:39:36.494 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-21T08:39:36.512 INFO:teuthology.orchestra.console.smithi137:Power on 2024-03-21T08:39:36.512 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-21T08:39:36.520 DEBUG:teuthology.orchestra.console.smithi136:power on output: Chassis Power Control: Up/On 2024-03-21T08:39:36.521 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:36.531 DEBUG:teuthology.orchestra.console.smithi137:power on output: Chassis Power Control: Up/On 2024-03-21T08:39:36.532 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:36.540 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off 2024-03-21T08:39:36.542 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-03-21T08:39:36.554 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-03-21T08:39:40.545 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:40.552 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:40.560 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:40.655 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-03-21T08:39:40.662 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-03-21T08:39:40.669 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-03-21T08:39:40.756 INFO:teuthology.orchestra.console.smithi063:Power on completed 2024-03-21T08:39:40.857 INFO:teuthology.provision.fog.smithi063:Waiting for deploy to finish 2024-03-21T08:39:44.666 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:44.673 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T08:39:44.775 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-21T08:39:44.782 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-21T08:39:44.876 INFO:teuthology.orchestra.console.smithi136:Power on completed 2024-03-21T08:39:44.883 INFO:teuthology.orchestra.console.smithi137:Power on completed 2024-03-21T08:39:44.977 INFO:teuthology.provision.fog.smithi136:Waiting for deploy to finish 2024-03-21T08:39:44.983 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish 2024-03-21T08:42:11.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:11.998 ERROR:teuthology.orchestra.connection:Error authenticating with smithi063.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-21T08:42:15.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:15.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:19.006 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-21T08:42:19.007 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-03-21T08:42:25.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:25.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:28.062 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-03-21T08:42:28.063 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-21T08:42:34.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:34.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:37.118 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-03-21T08:42:37.119 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-21T08:42:43.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:43.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:46.210 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-03-21T08:42:46.210 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-21T08:42:52.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:52.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:42:55.266 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-03-21T08:42:55.294 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-21T08:43:01.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:43:01.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:43:04.318 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-03-21T08:43:04.350 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-21T08:43:10.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:43:10.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:43:13.406 WARNING:teuthology.provision.fog.smithi136:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-03-21T08:43:16.001 WARNING:teuthology.provision.fog.smithi063:timed out 2024-03-21T08:43:19.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:43:22.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:10.322 WARNING:teuthology.provision.fog.smithi137:timed out 2024-03-21T08:44:16.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:16.556 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-21T08:44:16.557 DEBUG:teuthology.orchestra.run.smithi137:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-21T08:44:16.879 INFO:teuthology.provision.fog.smithi137:Node is ready 2024-03-21T08:44:16.879 DEBUG:teuthology.orchestra.run.smithi137:> hostname 2024-03-21T08:44:16.893 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com 2024-03-21T08:44:16.893 DEBUG:teuthology.orchestra.run.smithi137:> grep smithi137.front.sepia.ceph.com /etc/hosts 2024-03-21T08:44:16.948 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137 2024-03-21T08:44:16.949 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137 2024-03-21T08:44:17.071 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hosts 2024-03-21T08:44:17.100 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/172.21.15.137/172.21.15.137/g' /etc/hosts 2024-03-21T08:44:17.167 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hostname 2024-03-21T08:44:17.234 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137 2024-03-21T08:44:17.301 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-03-21T08:44:17.355 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-03-21T08:44:17.355 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-03-21T08:44:17.355 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-03-21T08:44:17.355 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:44:17.355 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-03-21T08:44:17.355 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:44:17.355 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:44:17.356 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-03-21T08:44:17.356 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:44:17.356 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:44:17.356 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:44:17.356 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:44:17.356 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:44:17.356 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:44:17.357 INFO:teuthology.provision.fog.smithi137:Deploy complete! 2024-03-21T08:44:17.359 DEBUG:teuthology.parallel:result is None 2024-03-21T08:44:19.411 WARNING:teuthology.provision.fog.smithi136:timed out 2024-03-21T08:44:22.005 WARNING:teuthology.provision.fog.smithi063:timed out 2024-03-21T08:44:25.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:25.674 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-21T08:44:25.674 DEBUG:teuthology.orchestra.run.smithi136:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-21T08:44:25.982 INFO:teuthology.provision.fog.smithi136:Node is ready 2024-03-21T08:44:25.982 DEBUG:teuthology.orchestra.run.smithi136:> hostname 2024-03-21T08:44:25.997 INFO:teuthology.orchestra.run.smithi136.stdout:smithi136.front.sepia.ceph.com 2024-03-21T08:44:25.998 DEBUG:teuthology.orchestra.run.smithi136:> grep smithi136.front.sepia.ceph.com /etc/hosts 2024-03-21T08:44:26.053 INFO:teuthology.orchestra.run.smithi136.stdout:172.21.15.136 smithi136.front.sepia.ceph.com smithi136 2024-03-21T08:44:26.053 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname smithi136 2024-03-21T08:44:26.179 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/smithi136.front.sepia.ceph.com/smithi136/g' /etc/hosts 2024-03-21T08:44:26.207 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/172.21.15.136/172.21.15.136/g' /etc/hosts 2024-03-21T08:44:26.275 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i -e 's/smithi136.front.sepia.ceph.com/smithi136/g' /etc/hostname 2024-03-21T08:44:26.342 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname smithi136 2024-03-21T08:44:26.409 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-03-21T08:44:26.464 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-03-21T08:44:26.464 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-03-21T08:44:26.464 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-03-21T08:44:26.464 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:44:26.464 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-03-21T08:44:26.464 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:44:26.464 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:44:26.465 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-03-21T08:44:26.465 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:44:26.465 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:44:26.465 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:44:26.465 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:44:26.465 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:44:26.465 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:44:26.465 INFO:teuthology.provision.fog.smithi136:Deploy complete! 2024-03-21T08:44:26.466 DEBUG:teuthology.parallel:result is None 2024-03-21T08:44:28.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:28.262 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-21T08:44:28.262 DEBUG:teuthology.orchestra.run.smithi063:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-21T08:44:28.616 INFO:teuthology.provision.fog.smithi063:Node is ready 2024-03-21T08:44:28.616 DEBUG:teuthology.orchestra.run.smithi063:> hostname 2024-03-21T08:44:28.632 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063.front.sepia.ceph.com 2024-03-21T08:44:28.632 DEBUG:teuthology.orchestra.run.smithi063:> grep smithi063.front.sepia.ceph.com /etc/hosts 2024-03-21T08:44:28.688 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63 smithi063.front.sepia.ceph.com smithi063 2024-03-21T08:44:28.689 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname smithi063 2024-03-21T08:44:28.811 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i -e 's/smithi063.front.sepia.ceph.com/smithi063/g' /etc/hosts 2024-03-21T08:44:28.844 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i -e 's/172.21.15.63/172.21.15.63/g' /etc/hosts 2024-03-21T08:44:28.916 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i -e 's/smithi063.front.sepia.ceph.com/smithi063/g' /etc/hostname 2024-03-21T08:44:28.987 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname smithi063 2024-03-21T08:44:29.057 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-03-21T08:44:29.112 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-03-21T08:44:29.112 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-03-21T08:44:29.112 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-03-21T08:44:29.112 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:44:29.112 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-03-21T08:44:29.112 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:44:29.112 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:44:29.113 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-03-21T08:44:29.113 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:44:29.113 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:44:29.113 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:44:29.113 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:44:29.113 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:44:29.113 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:44:29.113 INFO:teuthology.provision.fog.smithi063:Deploy complete! 2024-03-21T08:44:29.114 DEBUG:teuthology.parallel:result is None 2024-03-21T08:44:29.328 INFO:teuthology.lock.ops:Checking smithi063.front.sepia.ceph.com 2024-03-21T08:44:29.329 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-21T08:44:29.363 INFO:teuthology.lock.ops:Checking smithi136.front.sepia.ceph.com 2024-03-21T08:44:29.363 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-21T08:44:29.387 INFO:teuthology.lock.ops:Checking smithi137.front.sepia.ceph.com 2024-03-21T08:44:29.387 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-21T08:44:29.406 INFO:teuthology.lock.ops:Updating [smithi063.front.sepia.ceph.com]: set os type and version on server 2024-03-21T08:44:29.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-03-21T08:44:29.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:29.506 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-03-21T08:44:29.577 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-03-21T08:44:29.577 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-03-21T08:44:29.632 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-03-21T08:44:29.632 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-03-21T08:44:29.687 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-03-21T08:44:29.687 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-03-21T08:44:29.687 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:44:29.688 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:44:29.689 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-03-21T08:44:29.712 INFO:teuthology.lock.ops:Updating [smithi136.front.sepia.ceph.com]: set os type and version on server 2024-03-21T08:44:29.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-21T08:44:29.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:29.802 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-03-21T08:44:29.875 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-21T08:44:29.875 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-03-21T08:44:29.932 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-03-21T08:44:29.932 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-03-21T08:44:29.988 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-03-21T08:44:29.988 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:44:29.989 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:44:29.990 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-03-21T08:44:30.130 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: set os type and version on server 2024-03-21T08:44:30.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-21T08:44:30.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:30.220 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-03-21T08:44:30.388 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-21T08:44:30.389 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-03-21T08:44:30.443 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-03-21T08:44:30.443 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-03-21T08:44:30.498 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-03-21T08:44:30.498 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-03-21T08:44:30.498 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-03-21T08:44:30.498 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:44:30.498 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:44:30.499 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:44:30.500 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-03-21T08:44:30.529 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:44:30.634 INFO:teuthology.dispatcher.supervisor:Running job 7612159 2024-03-21T08:44:30.635 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612159 --name vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/pjd}} -- /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612159/orig.config.yaml 2024-03-21T08:44:30.638 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612159 2024-03-21T08:44:30.638 INFO:teuthology.dispatcher.supervisor:Job PID: 3810587 2024-03-21T08:44:30.638 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-21T08:46:30.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:48:30.710 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:50:30.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:52:30.880 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:54:30.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:56:31.027 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:58:31.095 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:00:31.168 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:02:31.245 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:04:31.326 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:06:31.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:08:31.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:10:31.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:12:31.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:12:31.720 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1