2024-03-01T17:30:26.830 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-01T17:30:26.830 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:30:27.002 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe1cc291c10>, signals=[15]) 2024-03-01T17:30:27.002 INFO:teuthology.lock.ops:Start node 'smithi131.front.sepia.ceph.com' reimaging 2024-03-01T17:30:27.002 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: reset os type and version on server 2024-03-01T17:30:27.002 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-03-01T17:30:27.027 INFO:teuthology.lock.ops:Start node 'smithi148.front.sepia.ceph.com' reimaging 2024-03-01T17:30:27.027 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: reset os type and version on server 2024-03-01T17:30:27.027 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-03-01T17:30:27.052 INFO:teuthology.lock.ops:Start node 'smithi150.front.sepia.ceph.com' reimaging 2024-03-01T17:30:27.052 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: reset os type and version on server 2024-03-01T17:30:27.052 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-03-01T17:30:27.143 DEBUG:teuthology.provision.fog.smithi131:Requesting image smithi_centos_9.stream (ID 10) 2024-03-01T17:30:27.169 DEBUG:teuthology.provision.fog.smithi148:Requesting image smithi_centos_9.stream (ID 10) 2024-03-01T17:30:27.172 DEBUG:teuthology.provision.fog.smithi150:Requesting image smithi_centos_9.stream (ID 10) 2024-03-01T17:30:27.190 INFO:teuthology.provision.fog.smithi131:Scheduling deploy of centos 9.stream 2024-03-01T17:30:27.210 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of centos 9.stream 2024-03-01T17:30:27.228 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of centos 9.stream 2024-03-01T17:30:27.809 INFO:teuthology.orchestra.console.smithi148:Power off 2024-03-01T17:30:27.809 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-01T17:30:27.823 INFO:teuthology.orchestra.console.smithi131:Power off 2024-03-01T17:30:27.823 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-01T17:30:27.831 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2024-03-01T17:30:27.832 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:27.849 INFO:teuthology.orchestra.console.smithi150:Power off 2024-03-01T17:30:27.849 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-01T17:30:27.857 DEBUG:teuthology.orchestra.console.smithi131:power off output: Chassis Power Control: Down/Off 2024-03-01T17:30:27.858 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:27.864 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-01T17:30:27.869 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2024-03-01T17:30:27.869 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:27.879 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-03-01T17:30:27.891 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-03-01T17:30:27.991 INFO:teuthology.orchestra.console.smithi150:Power off completed 2024-03-01T17:30:28.092 INFO:teuthology.orchestra.console.smithi150:Power on 2024-03-01T17:30:28.092 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-01T17:30:28.113 DEBUG:teuthology.orchestra.console.smithi150:power on output: Chassis Power Control: Up/On 2024-03-01T17:30:28.113 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:28.138 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-03-01T17:30:31.869 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:31.880 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:31.977 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-01T17:30:31.988 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-03-01T17:30:32.139 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:32.247 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-03-01T17:30:35.980 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:35.989 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:36.091 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-01T17:30:36.096 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2024-03-01T17:30:36.197 INFO:teuthology.orchestra.console.smithi131:Power off completed 2024-03-01T17:30:36.248 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:36.298 INFO:teuthology.orchestra.console.smithi131:Power on 2024-03-01T17:30:36.299 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-01T17:30:36.318 DEBUG:teuthology.orchestra.console.smithi131:power on output: Chassis Power Control: Up/On 2024-03-01T17:30:36.318 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:36.338 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2024-03-01T17:30:36.356 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-03-01T17:30:36.457 INFO:teuthology.orchestra.console.smithi150:Power on completed 2024-03-01T17:30:36.557 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish 2024-03-01T17:30:40.095 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:40.205 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2024-03-01T17:30:40.306 INFO:teuthology.orchestra.console.smithi148:Power off completed 2024-03-01T17:30:40.339 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:40.407 INFO:teuthology.orchestra.console.smithi148:Power on 2024-03-01T17:30:40.407 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-01T17:30:40.427 DEBUG:teuthology.orchestra.console.smithi148:power on output: Chassis Power Control: Up/On 2024-03-01T17:30:40.427 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:40.447 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2024-03-01T17:30:40.448 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is off 2024-03-01T17:30:44.448 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:44.454 DEBUG:teuthology.orchestra.console.smithi131:pexpect command: ipmitool -H smithi131.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:44.557 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2024-03-01T17:30:44.628 DEBUG:teuthology.orchestra.console.smithi131:check power output: Chassis Power is on 2024-03-01T17:30:44.729 INFO:teuthology.orchestra.console.smithi131:Power on completed 2024-03-01T17:30:44.829 INFO:teuthology.provision.fog.smithi131:Waiting for deploy to finish 2024-03-01T17:30:48.562 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-01T17:30:48.673 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-03-01T17:30:48.773 INFO:teuthology.orchestra.console.smithi148:Power on completed 2024-03-01T17:30:48.874 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish 2024-03-01T17:34:49.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:34:52.252 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-01T17:34:58.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:01.308 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-01T17:35:07.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:08.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:10.396 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-01T17:35:11.872 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-01T17:35:16.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:17.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:19.456 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-01T17:35:20.956 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-01T17:35:25.461 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:26.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:28.540 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-01T17:35:30.012 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-01T17:35:30.561 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:33.628 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-03-01T17:35:34.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:36.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:37.596 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-01T17:35:39.068 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-01T17:35:39.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:42.684 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-03-01T17:35:43.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:45.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:46.652 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-03-01T17:35:48.124 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-01T17:35:48.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:51.740 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-03-01T17:35:52.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:54.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:35:57.180 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-03-01T17:35:57.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:36:00.796 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-03-01T17:36:03.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:36:06.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:36:09.852 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-03-01T17:36:15.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:36:18.940 WARNING:teuthology.provision.fog.smithi131:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-03-01T17:36:24.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:36:52.680 WARNING:teuthology.provision.fog.smithi148:timed out 2024-03-01T17:36:58.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:36:59.038 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-01T17:36:59.039 DEBUG:teuthology.orchestra.run.smithi148:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-01T17:36:59.383 INFO:teuthology.provision.fog.smithi148:Node is ready 2024-03-01T17:36:59.383 DEBUG:teuthology.orchestra.run.smithi148:> hostname 2024-03-01T17:36:59.398 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148.front.sepia.ceph.com 2024-03-01T17:36:59.399 DEBUG:teuthology.orchestra.run.smithi148:> grep smithi148.front.sepia.ceph.com /etc/hosts 2024-03-01T17:36:59.457 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148 2024-03-01T17:36:59.457 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname smithi148 2024-03-01T17:36:59.560 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/smithi148.front.sepia.ceph.com/smithi148/g' /etc/hosts 2024-03-01T17:36:59.588 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/172.21.15.148/172.21.15.148/g' /etc/hosts 2024-03-01T17:36:59.659 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/smithi148.front.sepia.ceph.com/smithi148/g' /etc/hostname 2024-03-01T17:36:59.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname smithi148 2024-03-01T17:36:59.798 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-03-01T17:36:59.856 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-03-01T17:36:59.856 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-03-01T17:36:59.856 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-03-01T17:36:59.856 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-03-01T17:36:59.856 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-03-01T17:36:59.856 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-03-01T17:36:59.856 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-01T17:36:59.856 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-03-01T17:36:59.857 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-03-01T17:36:59.857 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-01T17:36:59.857 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-03-01T17:36:59.857 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-01T17:36:59.857 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-01T17:36:59.857 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-01T17:36:59.858 INFO:teuthology.provision.fog.smithi148:Deploy complete! 2024-03-01T17:36:59.859 DEBUG:teuthology.parallel:result is None 2024-03-01T17:37:03.189 WARNING:teuthology.provision.fog.smithi150:timed out 2024-03-01T17:37:09.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:37:09.487 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-01T17:37:09.488 DEBUG:teuthology.orchestra.run.smithi150:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-01T17:37:09.844 INFO:teuthology.provision.fog.smithi150:Node is ready 2024-03-01T17:37:09.845 DEBUG:teuthology.orchestra.run.smithi150:> hostname 2024-03-01T17:37:09.859 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com 2024-03-01T17:37:09.860 DEBUG:teuthology.orchestra.run.smithi150:> grep smithi150.front.sepia.ceph.com /etc/hosts 2024-03-01T17:37:09.915 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150 2024-03-01T17:37:09.915 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname smithi150 2024-03-01T17:37:10.016 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/smithi150.front.sepia.ceph.com/smithi150/g' /etc/hosts 2024-03-01T17:37:10.044 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/172.21.15.150/172.21.15.150/g' /etc/hosts 2024-03-01T17:37:10.111 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/smithi150.front.sepia.ceph.com/smithi150/g' /etc/hostname 2024-03-01T17:37:10.179 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname smithi150 2024-03-01T17:37:10.245 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-01T17:37:10.299 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-03-01T17:37:10.300 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-01T17:37:10.300 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-01T17:37:10.300 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-01T17:37:10.300 INFO:teuthology.provision.fog.smithi150:Deploy complete! 2024-03-01T17:37:10.300 DEBUG:teuthology.parallel:result is None 2024-03-01T17:37:24.963 WARNING:teuthology.provision.fog.smithi131:timed out 2024-03-01T17:37:30.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:37:31.219 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-01T17:37:31.219 DEBUG:teuthology.orchestra.run.smithi131:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-01T17:37:31.517 INFO:teuthology.provision.fog.smithi131:Node is ready 2024-03-01T17:37:31.517 DEBUG:teuthology.orchestra.run.smithi131:> hostname 2024-03-01T17:37:31.532 INFO:teuthology.orchestra.run.smithi131.stdout:smithi131.front.sepia.ceph.com 2024-03-01T17:37:31.532 DEBUG:teuthology.orchestra.run.smithi131:> grep smithi131.front.sepia.ceph.com /etc/hosts 2024-03-01T17:37:31.587 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131 smithi131.front.sepia.ceph.com smithi131 2024-03-01T17:37:31.587 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131 2024-03-01T17:37:31.688 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hosts 2024-03-01T17:37:31.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/172.21.15.131/172.21.15.131/g' /etc/hosts 2024-03-01T17:37:31.782 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i -e 's/smithi131.front.sepia.ceph.com/smithi131/g' /etc/hostname 2024-03-01T17:37:31.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname smithi131 2024-03-01T17:37:31.915 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-01T17:37:31.970 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-03-01T17:37:31.971 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-01T17:37:31.971 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-01T17:37:31.971 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-01T17:37:31.971 INFO:teuthology.provision.fog.smithi131:Deploy complete! 2024-03-01T17:37:31.971 DEBUG:teuthology.parallel:result is None 2024-03-01T17:37:32.176 INFO:teuthology.lock.ops:Checking smithi131.front.sepia.ceph.com 2024-03-01T17:37:32.176 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-01T17:37:32.200 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com 2024-03-01T17:37:32.200 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-01T17:37:32.216 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com 2024-03-01T17:37:32.216 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-01T17:37:32.234 INFO:teuthology.lock.ops:Updating [smithi131.front.sepia.ceph.com]: set os type and version on server 2024-03-01T17:37:32.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-03-01T17:37:32.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:37:32.319 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-03-01T17:37:32.381 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-03-01T17:37:32.381 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-03-01T17:37:32.436 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-03-01T17:37:32.436 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-03-01T17:37:32.490 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-01T17:37:32.491 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-03-01T17:37:32.491 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-01T17:37:32.491 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-01T17:37:32.491 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-01T17:37:32.491 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-03-01T17:37:32.516 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server 2024-03-01T17:37:32.516 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-01T17:37:32.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:37:32.601 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-03-01T17:37:32.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-03-01T17:37:32.766 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-03-01T17:37:32.820 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-03-01T17:37:32.821 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-01T17:37:32.876 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-01T17:37:32.877 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-01T17:37:32.877 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-03-01T17:37:32.899 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server 2024-03-01T17:37:32.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-01T17:37:32.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-01T17:37:32.992 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-03-01T17:37:33.162 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-03-01T17:37:33.163 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-03-01T17:37:33.217 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-03-01T17:37:33.218 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-03-01T17:37:33.273 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-03-01T17:37:33.273 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-03-01T17:37:33.273 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-03-01T17:37:33.273 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-03-01T17:37:33.273 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-03-01T17:37:33.273 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-03-01T17:37:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-01T17:37:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-03-01T17:37:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-03-01T17:37:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-01T17:37:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-03-01T17:37:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-01T17:37:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-01T17:37:33.274 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-01T17:37:33.274 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-03-01T17:37:33.300 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:37:33.390 INFO:teuthology.dispatcher.supervisor:Running job 7577665 2024-03-01T17:37:33.390 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-03-01_17:28:49-fs-main-2024feb28-testing-default-smithi/7577665 --name rishabh-2024-03-01_17:28:49-fs-main-2024feb28-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} -- /home/teuthworker/archive/rishabh-2024-03-01_17:28:49-fs-main-2024feb28-testing-default-smithi/7577665/orig.config.yaml 2024-03-01T17:37:33.393 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2024-03-01_17:28:49-fs-main-2024feb28-testing-default-smithi/7577665 2024-03-01T17:37:33.393 INFO:teuthology.dispatcher.supervisor:Job PID: 2221291 2024-03-01T17:37:33.393 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-01T17:39:33.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:41:33.649 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:43:33.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:45:33.985 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:47:34.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:49:34.341 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:51:34.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:53:34.721 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:55:34.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:57:35.153 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T17:59:35.317 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:01:35.481 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:03:35.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:05:35.777 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:07:35.933 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:09:36.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:11:36.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:13:36.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:15:36.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:17:36.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-01T18:17:36.874 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1