2024-03-12T03:42:39.501 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-12T03:42:39.501 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:42:39.679 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd938b0fc10>, signals=[15]) 2024-03-12T03:42:39.680 INFO:teuthology.lock.ops:Start node 'smithi092.front.sepia.ceph.com' reimaging 2024-03-12T03:42:39.680 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: reset os type and version on server 2024-03-12T03:42:39.680 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-03-12T03:42:39.703 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging 2024-03-12T03:42:39.703 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server 2024-03-12T03:42:39.704 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-03-12T03:42:39.719 INFO:teuthology.lock.ops:Start node 'smithi196.front.sepia.ceph.com' reimaging 2024-03-12T03:42:39.719 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: reset os type and version on server 2024-03-12T03:42:39.720 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-03-12T03:42:39.810 DEBUG:teuthology.provision.fog.smithi092:Requesting image smithi_centos_9.stream (ID 10) 2024-03-12T03:42:39.819 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_centos_9.stream (ID 10) 2024-03-12T03:42:39.848 INFO:teuthology.provision.fog.smithi092:Scheduling deploy of centos 9.stream 2024-03-12T03:42:39.853 DEBUG:teuthology.provision.fog.smithi196:Requesting image smithi_centos_9.stream (ID 10) 2024-03-12T03:42:39.864 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 9.stream 2024-03-12T03:42:39.888 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of centos 9.stream 2024-03-12T03:42:40.138 INFO:teuthology.orchestra.console.smithi159:Power off 2024-03-12T03:42:40.138 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T03:42:40.150 INFO:teuthology.orchestra.console.smithi092:Power off 2024-03-12T03:42:40.151 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T03:42:40.160 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-03-12T03:42:40.160 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:40.175 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-03-12T03:42:40.175 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:40.188 INFO:teuthology.orchestra.console.smithi196:Power off 2024-03-12T03:42:40.188 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-12T03:42:40.195 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-03-12T03:42:40.199 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-12T03:42:40.209 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-03-12T03:42:40.209 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:40.230 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-03-12T03:42:40.296 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-03-12T03:42:40.331 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-03-12T03:42:40.397 INFO:teuthology.orchestra.console.smithi159:Power on 2024-03-12T03:42:40.398 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-12T03:42:40.419 DEBUG:teuthology.orchestra.console.smithi159:power on output: Chassis Power Control: Up/On 2024-03-12T03:42:40.420 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:40.432 INFO:teuthology.orchestra.console.smithi196:Power on 2024-03-12T03:42:40.432 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-12T03:42:40.443 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-03-12T03:42:40.454 DEBUG:teuthology.orchestra.console.smithi196:power on output: Chassis Power Control: Up/On 2024-03-12T03:42:40.454 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:40.475 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-03-12T03:42:44.203 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:44.312 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-03-12T03:42:44.413 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-03-12T03:42:44.443 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:44.476 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:44.514 INFO:teuthology.orchestra.console.smithi092:Power on 2024-03-12T03:42:44.515 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-12T03:42:44.535 DEBUG:teuthology.orchestra.console.smithi092:power on output: Chassis Power Control: Up/On 2024-03-12T03:42:44.535 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:44.554 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-03-12T03:42:44.555 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-03-12T03:42:44.583 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-03-12T03:42:44.655 INFO:teuthology.orchestra.console.smithi159:Power on completed 2024-03-12T03:42:44.756 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2024-03-12T03:42:48.559 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:48.585 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:48.669 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-03-12T03:42:48.708 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-12T03:42:48.809 INFO:teuthology.orchestra.console.smithi196:Power on completed 2024-03-12T03:42:48.909 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2024-03-12T03:42:52.671 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-12T03:42:52.779 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-12T03:42:52.881 INFO:teuthology.orchestra.console.smithi092:Power on completed 2024-03-12T03:42:52.981 INFO:teuthology.provision.fog.smithi092:Waiting for deploy to finish 2024-03-12T03:45:20.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:23.070 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-12T03:45:24.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:27.166 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:45:29.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:31.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:32.126 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-12T03:45:33.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:34.078 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-03-12T03:45:36.222 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:45:38.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:40.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:41.182 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-12T03:45:42.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:43.134 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-03-12T03:45:45.282 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:45:47.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:49.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:50.238 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-12T03:45:51.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:52.194 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-03-12T03:45:54.334 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:45:56.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:58.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:45:59.294 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-12T03:46:00.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:01.282 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-03-12T03:46:03.394 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:46:05.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:07.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:08.350 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-12T03:46:09.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:10.366 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-03-12T03:46:12.446 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:46:14.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:16.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:17.406 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-03-12T03:46:18.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:19.422 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-03-12T03:46:21.502 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:46:23.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:25.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:27.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:45.922 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:46:51.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:46:54.978 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:47:00.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:04.062 WARNING:teuthology.provision.fog.smithi092:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-12T03:47:10.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:10.401 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-12T03:47:10.402 DEBUG:teuthology.orchestra.run.smithi092:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-12T03:47:23.412 WARNING:teuthology.provision.fog.smithi196:timed out 2024-03-12T03:47:25.428 WARNING:teuthology.provision.fog.smithi159:timed out 2024-03-12T03:47:29.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:29.805 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-12T03:47:29.806 DEBUG:teuthology.orchestra.run.smithi196:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-12T03:47:30.119 INFO:teuthology.provision.fog.smithi196:Node is ready 2024-03-12T03:47:30.119 DEBUG:teuthology.orchestra.run.smithi196:> hostname 2024-03-12T03:47:30.134 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com 2024-03-12T03:47:30.134 DEBUG:teuthology.orchestra.run.smithi196:> grep smithi196.front.sepia.ceph.com /etc/hosts 2024-03-12T03:47:30.189 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196 2024-03-12T03:47:30.190 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2024-03-12T03:47:30.301 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hosts 2024-03-12T03:47:30.329 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/172.21.15.196/172.21.15.196/g' /etc/hosts 2024-03-12T03:47:30.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hostname 2024-03-12T03:47:30.465 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2024-03-12T03:47:30.532 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:30.587 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:30.588 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:30.588 INFO:teuthology.provision.fog.smithi196:Deploy complete! 2024-03-12T03:47:30.590 DEBUG:teuthology.parallel:result is None 2024-03-12T03:47:30.750 INFO:teuthology.provision.fog.smithi092:Node is ready 2024-03-12T03:47:30.751 DEBUG:teuthology.orchestra.run.smithi092:> hostname 2024-03-12T03:47:30.769 INFO:teuthology.orchestra.run.smithi092.stdout:smithi092.front.sepia.ceph.com 2024-03-12T03:47:30.769 DEBUG:teuthology.orchestra.run.smithi092:> grep smithi092.front.sepia.ceph.com /etc/hosts 2024-03-12T03:47:30.826 INFO:teuthology.orchestra.run.smithi092.stdout:172.21.15.92 smithi092.front.sepia.ceph.com smithi092 2024-03-12T03:47:30.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname smithi092 2024-03-12T03:47:30.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/smithi092.front.sepia.ceph.com/smithi092/g' /etc/hosts 2024-03-12T03:47:30.986 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/172.21.15.92/172.21.15.92/g' /etc/hosts 2024-03-12T03:47:31.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i -e 's/smithi092.front.sepia.ceph.com/smithi092/g' /etc/hostname 2024-03-12T03:47:31.128 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname smithi092 2024-03-12T03:47:31.196 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:31.253 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:31.254 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:31.254 INFO:teuthology.provision.fog.smithi092:Deploy complete! 2024-03-12T03:47:31.255 DEBUG:teuthology.parallel:result is None 2024-03-12T03:47:31.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:31.729 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-12T03:47:31.729 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-12T03:47:32.047 INFO:teuthology.provision.fog.smithi159:Node is ready 2024-03-12T03:47:32.047 DEBUG:teuthology.orchestra.run.smithi159:> hostname 2024-03-12T03:47:32.062 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2024-03-12T03:47:32.062 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi159.front.sepia.ceph.com /etc/hosts 2024-03-12T03:47:32.118 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2024-03-12T03:47:32.119 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2024-03-12T03:47:32.226 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hosts 2024-03-12T03:47:32.254 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.159/172.21.15.159/g' /etc/hosts 2024-03-12T03:47:32.322 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hostname 2024-03-12T03:47:32.390 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2024-03-12T03:47:32.456 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:32.511 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:32.512 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:32.512 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:32.512 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:32.512 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:32.512 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2024-03-12T03:47:32.512 DEBUG:teuthology.parallel:result is None 2024-03-12T03:47:32.724 INFO:teuthology.lock.ops:Checking smithi092.front.sepia.ceph.com 2024-03-12T03:47:32.724 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-12T03:47:32.745 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2024-03-12T03:47:32.745 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-12T03:47:32.761 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com 2024-03-12T03:47:32.761 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-12T03:47:32.780 INFO:teuthology.lock.ops:Updating [smithi092.front.sepia.ceph.com]: set os type and version on server 2024-03-12T03:47:32.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-12T03:47:32.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:32.881 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-03-12T03:47:32.949 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-12T03:47:32.949 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-03-12T03:47:33.005 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-03-12T03:47:33.006 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-03-12T03:47:33.061 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:33.062 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:33.063 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:33.063 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-03-12T03:47:33.087 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2024-03-12T03:47:33.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-12T03:47:33.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:33.173 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-03-12T03:47:33.241 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-12T03:47:33.241 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-03-12T03:47:33.296 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-03-12T03:47:33.296 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-03-12T03:47:33.351 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-03-12T03:47:33.351 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-03-12T03:47:33.351 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-03-12T03:47:33.351 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:33.351 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-03-12T03:47:33.351 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:33.352 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:33.352 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:33.352 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:33.352 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:33.352 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:33.352 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:33.352 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:33.352 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:33.353 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-03-12T03:47:33.370 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server 2024-03-12T03:47:33.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-12T03:47:33.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-12T03:47:33.462 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-03-12T03:47:33.531 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-12T03:47:33.531 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-03-12T03:47:33.587 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-03-12T03:47:33.588 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-03-12T03:47:33.643 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-03-12T03:47:33.643 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-03-12T03:47:33.643 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-03-12T03:47:33.643 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-03-12T03:47:33.643 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-03-12T03:47:33.643 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-03-12T03:47:33.643 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-12T03:47:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-03-12T03:47:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-03-12T03:47:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-12T03:47:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-03-12T03:47:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-12T03:47:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-12T03:47:33.644 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-12T03:47:33.644 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-03-12T03:47:33.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:47:33.761 INFO:teuthology.dispatcher.supervisor:Running job 7591351 2024-03-12T03:47:33.762 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591351 --name teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/test_o_trunc}} -- /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591351/orig.config.yaml 2024-03-12T03:47:33.765 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591351 2024-03-12T03:47:33.765 INFO:teuthology.dispatcher.supervisor:Job PID: 1540320 2024-03-12T03:47:33.765 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-12T03:49:33.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:51:34.031 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:53:34.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:55:34.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:57:34.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T03:59:34.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:01:34.883 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:03:35.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:05:35.221 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:07:35.341 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:09:35.514 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:11:35.686 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:13:35.859 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:15:36.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:17:36.178 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:19:36.315 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:21:36.483 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:23:36.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:25:36.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:27:36.998 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:29:37.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:31:37.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-12T04:31:37.358 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1