2024-04-18T09:24:36.424 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-18T09:24:36.424 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T09:24:36.599 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7208424c10>, signals=[15]) 2024-04-18T09:24:36.600 INFO:teuthology.lock.ops:Start node 'smithi031.front.sepia.ceph.com' reimaging 2024-04-18T09:24:36.600 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T09:24:36.600 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-04-18T09:24:36.623 INFO:teuthology.lock.ops:Start node 'smithi165.front.sepia.ceph.com' reimaging 2024-04-18T09:24:36.623 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T09:24:36.623 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-04-18T09:24:36.647 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging 2024-04-18T09:24:36.647 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T09:24:36.647 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-04-18T09:24:36.738 DEBUG:teuthology.provision.fog.smithi031:Requesting image smithi_centos_9.stream (ID 10) 2024-04-18T09:24:36.756 DEBUG:teuthology.provision.fog.smithi165:Requesting image smithi_centos_9.stream (ID 10) 2024-04-18T09:24:36.768 DEBUG:teuthology.provision.fog.smithi174:Requesting image smithi_centos_9.stream (ID 10) 2024-04-18T09:24:36.789 INFO:teuthology.provision.fog.smithi031:Scheduling deploy of centos 9.stream 2024-04-18T09:24:36.795 INFO:teuthology.provision.fog.smithi165:Scheduling deploy of centos 9.stream 2024-04-18T09:24:36.817 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 9.stream 2024-04-18T09:24:37.019 INFO:teuthology.orchestra.console.smithi165:Power off 2024-04-18T09:24:37.019 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T09:24:37.033 INFO:teuthology.orchestra.console.smithi174:Power off 2024-04-18T09:24:37.034 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T09:24:37.048 INFO:teuthology.orchestra.console.smithi031:Power off 2024-04-18T09:24:37.049 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T09:24:37.057 DEBUG:teuthology.orchestra.console.smithi165:power off output: Chassis Power Control: Down/Off 2024-04-18T09:24:37.057 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:37.068 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2024-04-18T09:24:37.068 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:37.078 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2024-04-18T09:24:37.078 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:37.087 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-04-18T09:24:37.092 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-04-18T09:24:37.102 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-18T09:24:41.090 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:41.097 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:41.105 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:41.199 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-04-18T09:24:41.206 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-04-18T09:24:41.213 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-18T09:24:45.202 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:45.210 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:45.222 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:45.312 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-04-18T09:24:45.324 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-04-18T09:24:45.330 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2024-04-18T09:24:45.425 INFO:teuthology.orchestra.console.smithi174:Power off completed 2024-04-18T09:24:45.431 INFO:teuthology.orchestra.console.smithi031:Power off completed 2024-04-18T09:24:45.526 INFO:teuthology.orchestra.console.smithi174:Power on 2024-04-18T09:24:45.526 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T09:24:45.534 INFO:teuthology.orchestra.console.smithi031:Power on 2024-04-18T09:24:45.534 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T09:24:45.548 DEBUG:teuthology.orchestra.console.smithi174:power on output: Chassis Power Control: Up/On 2024-04-18T09:24:45.548 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:45.556 DEBUG:teuthology.orchestra.console.smithi031:power on output: Chassis Power Control: Up/On 2024-04-18T09:24:45.556 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:45.569 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-04-18T09:24:45.578 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2024-04-18T09:24:49.317 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:49.426 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off 2024-04-18T09:24:49.527 INFO:teuthology.orchestra.console.smithi165:Power off completed 2024-04-18T09:24:49.570 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:49.578 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:49.628 INFO:teuthology.orchestra.console.smithi165:Power on 2024-04-18T09:24:49.628 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T09:24:49.653 DEBUG:teuthology.orchestra.console.smithi165:power on output: Chassis Power Control: Up/On 2024-04-18T09:24:49.654 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:49.679 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is off 2024-04-18T09:24:49.680 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-04-18T09:24:49.687 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2024-04-18T09:24:53.682 DEBUG:teuthology.orchestra.console.smithi165:pexpect command: ipmitool -H smithi165.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:53.690 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:53.698 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T09:24:53.793 DEBUG:teuthology.orchestra.console.smithi165:check power output: Chassis Power is on 2024-04-18T09:24:53.800 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-04-18T09:24:53.806 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-18T09:24:53.894 INFO:teuthology.orchestra.console.smithi165:Power on completed 2024-04-18T09:24:53.900 INFO:teuthology.orchestra.console.smithi174:Power on completed 2024-04-18T09:24:53.907 INFO:teuthology.orchestra.console.smithi031:Power on completed 2024-04-18T09:24:53.995 INFO:teuthology.provision.fog.smithi165:Waiting for deploy to finish 2024-04-18T09:24:54.001 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2024-04-18T09:24:54.008 INFO:teuthology.provision.fog.smithi031:Waiting for deploy to finish 2024-04-18T09:27:09.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:12.958 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-04-18T09:27:18.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:22.046 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-04-18T09:27:24.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:28.030 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-18T09:27:28.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:31.106 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-04-18T09:27:34.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:37.090 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-18T09:27:37.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:40.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:40.158 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-04-18T09:27:43.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:43.102 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-18T09:27:46.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:46.178 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-18T09:27:49.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:49.214 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-04-18T09:27:52.162 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-18T09:27:52.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:55.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:27:55.266 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-18T09:27:58.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:01.250 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-18T09:28:01.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:04.322 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-18T09:28:07.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:10.327 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:10.334 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-18T09:28:13.406 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-18T09:28:16.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:19.394 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-18T09:28:19.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:22.462 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-18T09:28:25.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:28.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:28.478 WARNING:teuthology.provision.fog.smithi165:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-18T09:28:34.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:28:55.241 WARNING:teuthology.provision.fog.smithi031:timed out 2024-04-18T09:29:01.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:29:01.566 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T09:29:01.567 DEBUG:teuthology.orchestra.run.smithi031:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T09:29:01.879 INFO:teuthology.provision.fog.smithi031:Node is ready 2024-04-18T09:29:01.879 DEBUG:teuthology.orchestra.run.smithi031:> hostname 2024-04-18T09:29:01.894 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031.front.sepia.ceph.com 2024-04-18T09:29:01.895 DEBUG:teuthology.orchestra.run.smithi031:> grep smithi031.front.sepia.ceph.com /etc/hosts 2024-04-18T09:29:01.951 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31 smithi031.front.sepia.ceph.com smithi031 2024-04-18T09:29:01.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname smithi031 2024-04-18T09:29:02.055 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/smithi031.front.sepia.ceph.com/smithi031/g' /etc/hosts 2024-04-18T09:29:02.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/172.21.15.31/172.21.15.31/g' /etc/hosts 2024-04-18T09:29:02.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/smithi031.front.sepia.ceph.com/smithi031/g' /etc/hostname 2024-04-18T09:29:02.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname smithi031 2024-04-18T09:29:02.299 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-04-18T09:29:02.354 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-04-18T09:29:02.354 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-04-18T09:29:02.354 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-04-18T09:29:02.354 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-04-18T09:29:02.354 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-04-18T09:29:02.354 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-04-18T09:29:02.354 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T09:29:02.355 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-04-18T09:29:02.355 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-04-18T09:29:02.355 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T09:29:02.355 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-04-18T09:29:02.355 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T09:29:02.355 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T09:29:02.355 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T09:29:02.356 INFO:teuthology.provision.fog.smithi031:Deploy complete! 2024-04-18T09:29:02.357 DEBUG:teuthology.parallel:result is None 2024-04-18T09:29:28.495 WARNING:teuthology.provision.fog.smithi174:timed out 2024-04-18T09:29:34.490 WARNING:teuthology.provision.fog.smithi165:timed out 2024-04-18T09:29:34.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:29:34.892 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T09:29:34.892 DEBUG:teuthology.orchestra.run.smithi174:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T09:29:35.232 INFO:teuthology.provision.fog.smithi174:Node is ready 2024-04-18T09:29:35.232 DEBUG:teuthology.orchestra.run.smithi174:> hostname 2024-04-18T09:29:35.250 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174.front.sepia.ceph.com 2024-04-18T09:29:35.251 DEBUG:teuthology.orchestra.run.smithi174:> grep smithi174.front.sepia.ceph.com /etc/hosts 2024-04-18T09:29:35.306 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2024-04-18T09:29:35.306 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2024-04-18T09:29:35.422 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hosts 2024-04-18T09:29:35.455 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/172.21.15.174/172.21.15.174/g' /etc/hosts 2024-04-18T09:29:35.523 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174.front.sepia.ceph.com/smithi174/g' /etc/hostname 2024-04-18T09:29:35.590 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2024-04-18T09:29:35.659 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-04-18T09:29:35.714 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-04-18T09:29:35.714 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-04-18T09:29:35.714 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-04-18T09:29:35.714 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-04-18T09:29:35.714 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-04-18T09:29:35.714 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-04-18T09:29:35.714 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T09:29:35.714 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-04-18T09:29:35.715 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-04-18T09:29:35.715 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T09:29:35.715 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-04-18T09:29:35.715 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T09:29:35.715 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T09:29:35.715 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T09:29:35.715 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2024-04-18T09:29:35.716 DEBUG:teuthology.parallel:result is None 2024-04-18T09:29:40.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:29:40.755 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T09:29:40.756 DEBUG:teuthology.orchestra.run.smithi165:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T09:29:41.097 INFO:teuthology.provision.fog.smithi165:Node is ready 2024-04-18T09:29:41.097 DEBUG:teuthology.orchestra.run.smithi165:> hostname 2024-04-18T09:29:41.112 INFO:teuthology.orchestra.run.smithi165.stdout:smithi165.front.sepia.ceph.com 2024-04-18T09:29:41.112 DEBUG:teuthology.orchestra.run.smithi165:> grep smithi165.front.sepia.ceph.com /etc/hosts 2024-04-18T09:29:41.167 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165 smithi165.front.sepia.ceph.com smithi165 2024-04-18T09:29:41.168 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname smithi165 2024-04-18T09:29:41.264 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/smithi165.front.sepia.ceph.com/smithi165/g' /etc/hosts 2024-04-18T09:29:41.292 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/172.21.15.165/172.21.15.165/g' /etc/hosts 2024-04-18T09:29:41.360 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i -e 's/smithi165.front.sepia.ceph.com/smithi165/g' /etc/hostname 2024-04-18T09:29:41.428 DEBUG:teuthology.orchestra.run.smithi165:> sudo hostname smithi165 2024-04-18T09:29:41.494 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-04-18T09:29:41.549 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-04-18T09:29:41.549 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-04-18T09:29:41.549 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T09:29:41.550 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T09:29:41.551 INFO:teuthology.provision.fog.smithi165:Deploy complete! 2024-04-18T09:29:41.551 DEBUG:teuthology.parallel:result is None 2024-04-18T09:29:41.774 INFO:teuthology.lock.ops:Checking smithi031.front.sepia.ceph.com 2024-04-18T09:29:41.774 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T09:29:41.797 INFO:teuthology.lock.ops:Checking smithi165.front.sepia.ceph.com 2024-04-18T09:29:41.797 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T09:29:41.814 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2024-04-18T09:29:41.814 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T09:29:41.832 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: set os type and version on server 2024-04-18T09:29:41.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-18T09:29:41.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:29:41.942 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-04-18T09:29:42.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-18T09:29:42.125 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-04-18T09:29:42.181 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-04-18T09:29:42.182 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-04-18T09:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-04-18T09:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-04-18T09:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-04-18T09:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-04-18T09:29:42.237 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T09:29:42.238 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T09:29:42.239 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-04-18T09:29:42.479 INFO:teuthology.lock.ops:Updating [smithi165.front.sepia.ceph.com]: set os type and version on server 2024-04-18T09:29:42.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-18T09:29:42.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:29:42.575 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-04-18T09:29:42.642 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-18T09:29:42.642 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-04-18T09:29:42.698 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-04-18T09:29:42.698 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T09:29:42.754 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-04-18T09:29:42.755 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T09:29:42.755 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T09:29:42.755 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T09:29:42.755 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-04-18T09:29:42.789 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2024-04-18T09:29:42.789 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-04-18T09:29:42.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T09:29:42.878 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-04-18T09:29:42.946 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-04-18T09:29:42.946 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-04-18T09:29:43.002 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-04-18T09:29:43.002 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-04-18T09:29:43.058 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-04-18T09:29:43.059 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T09:29:43.059 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-04-18T09:29:43.059 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T09:29:43.059 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T09:29:43.059 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T09:29:43.059 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-04-18T09:29:43.090 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T09:29:43.193 INFO:teuthology.dispatcher.supervisor:Running job 7660891 2024-04-18T09:29:43.194 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-17_20:24:14-fs-main-distro-default-smithi/7660891 --name teuthology-2024-04-17_20:24:14-fs-main-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} -- /home/teuthworker/archive/teuthology-2024-04-17_20:24:14-fs-main-distro-default-smithi/7660891/orig.config.yaml 2024-04-18T09:29:43.197 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-17_20:24:14-fs-main-distro-default-smithi/7660891 2024-04-18T09:29:43.197 INFO:teuthology.dispatcher.supervisor:Job PID: 3061638 2024-04-18T09:29:43.197 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-18T09:31:43.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T09:33:43.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T09:35:43.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T09:37:43.815 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T09:39:43.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T09:39:44.058 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1