2024-04-18T10:17:36.001 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-18T10:17:36.001 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T10:17:36.173 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa295086c10>, signals=[15]) 2024-04-18T10:17:36.174 INFO:teuthology.lock.ops:Start node 'smithi090.front.sepia.ceph.com' reimaging 2024-04-18T10:17:36.174 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T10:17:36.174 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-18T10:17:36.193 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging 2024-04-18T10:17:36.193 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T10:17:36.193 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-04-18T10:17:36.218 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging 2024-04-18T10:17:36.218 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T10:17:36.218 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-04-18T10:17:36.307 DEBUG:teuthology.provision.fog.smithi090:Requesting image smithi_centos_9.stream (ID 10) 2024-04-18T10:17:36.324 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_centos_9.stream (ID 10) 2024-04-18T10:17:36.332 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_centos_9.stream (ID 10) 2024-04-18T10:17:36.360 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of centos 9.stream 2024-04-18T10:17:36.374 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of centos 9.stream 2024-04-18T10:17:36.377 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 9.stream 2024-04-18T10:17:36.808 INFO:teuthology.orchestra.console.smithi159:Power off 2024-04-18T10:17:36.808 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T10:17:36.825 INFO:teuthology.orchestra.console.smithi090:Power off 2024-04-18T10:17:36.825 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T10:17:36.837 INFO:teuthology.orchestra.console.smithi118:Power off 2024-04-18T10:17:36.837 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T10:17:36.845 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-04-18T10:17:36.845 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:36.854 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-04-18T10:17:36.854 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:36.862 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2024-04-18T10:17:36.862 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:36.871 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-04-18T10:17:36.875 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-04-18T10:17:36.883 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-04-18T10:17:36.972 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-04-18T10:17:36.976 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-04-18T10:17:36.983 INFO:teuthology.orchestra.console.smithi118:Power off completed 2024-04-18T10:17:37.074 INFO:teuthology.orchestra.console.smithi159:Power on 2024-04-18T10:17:37.074 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T10:17:37.082 INFO:teuthology.orchestra.console.smithi090:Power on 2024-04-18T10:17:37.082 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T10:17:37.090 INFO:teuthology.orchestra.console.smithi118:Power on 2024-04-18T10:17:37.090 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T10:17:37.098 DEBUG:teuthology.orchestra.console.smithi159:power on output: Chassis Power Control: Up/On 2024-04-18T10:17:37.099 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:37.107 DEBUG:teuthology.orchestra.console.smithi090:power on output: Chassis Power Control: Up/On 2024-04-18T10:17:37.108 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:37.116 DEBUG:teuthology.orchestra.console.smithi118:power on output: Chassis Power Control: Up/On 2024-04-18T10:17:37.116 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:37.124 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-04-18T10:17:37.129 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-04-18T10:17:37.137 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-04-18T10:17:41.126 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:41.133 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:41.141 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:41.236 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-04-18T10:17:41.243 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-04-18T10:17:41.249 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-04-18T10:17:41.337 INFO:teuthology.orchestra.console.smithi159:Power on completed 2024-04-18T10:17:41.437 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2024-04-18T10:17:45.247 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:45.255 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T10:17:45.357 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-18T10:17:45.458 INFO:teuthology.orchestra.console.smithi090:Power on completed 2024-04-18T10:17:45.484 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2024-04-18T10:17:45.559 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish 2024-04-18T10:17:45.584 INFO:teuthology.orchestra.console.smithi118:Power on completed 2024-04-18T10:17:45.685 INFO:teuthology.provision.fog.smithi118:Waiting for deploy to finish 2024-04-18T10:22:49.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:22:53.058 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:22:59.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:02.142 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:23:08.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:11.198 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:23:17.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:20.254 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:23:26.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:29.310 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:23:35.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:35.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:38.366 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-18T10:23:38.367 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:23:44.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:44.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:47.422 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-18T10:23:47.423 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:23:53.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:53.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:23:56.478 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-18T10:24:02.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:05.534 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-18T10:24:11.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:11.806 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:24:14.590 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-18T10:24:16.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:17.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:19.550 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:24:20.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:20.862 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:24:23.650 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-18T10:24:25.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:26.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:28.610 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:24:29.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:29.918 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-18T10:24:32.734 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-18T10:24:34.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:35.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:36.196 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T10:24:36.197 DEBUG:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T10:24:37.694 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:24:38.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:43.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:46.750 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:24:52.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:24:55.806 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:24:56.543 INFO:teuthology.provision.fog.smithi118:Node is ready 2024-04-18T10:24:56.543 DEBUG:teuthology.orchestra.run.smithi118:> hostname 2024-04-18T10:24:56.562 INFO:teuthology.orchestra.run.smithi118.stdout:smithi046 2024-04-18T10:24:56.562 DEBUG:teuthology.orchestra.run.smithi118:> grep smithi046 /etc/hosts 2024-04-18T10:24:56.619 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-04-18T10:24:56.619 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2024-04-18T10:24:56.778 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi046/smithi118/g' /etc/hosts 2024-04-18T10:24:56.810 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/172.21.15.46/172.21.15.118/g' /etc/hosts 2024-04-18T10:24:56.879 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi046/smithi118/g' /etc/hostname 2024-04-18T10:24:56.949 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2024-04-18T10:24:57.024 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T10:24:57.079 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-04-18T10:24:57.080 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T10:24:57.080 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T10:24:57.080 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T10:24:57.080 INFO:teuthology.provision.fog.smithi118:Deploy complete! 2024-04-18T10:24:57.082 DEBUG:teuthology.parallel:result is None 2024-04-18T10:25:01.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:25:04.894 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:25:10.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:25:13.982 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:25:19.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:25:23.070 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:25:29.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:25:38.746 WARNING:teuthology.provision.fog.smithi090:timed out 2024-04-18T10:25:44.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:25:45.064 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T10:25:45.064 DEBUG:teuthology.orchestra.run.smithi090:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T10:25:47.550 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:25:50.417 INFO:teuthology.provision.fog.smithi090:Node is ready 2024-04-18T10:25:50.418 DEBUG:teuthology.orchestra.run.smithi090:> hostname 2024-04-18T10:25:50.435 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090.front.sepia.ceph.com 2024-04-18T10:25:50.436 DEBUG:teuthology.orchestra.run.smithi090:> grep smithi090.front.sepia.ceph.com /etc/hosts 2024-04-18T10:25:50.493 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90 smithi090.front.sepia.ceph.com smithi090 2024-04-18T10:25:50.493 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090 2024-04-18T10:25:50.634 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi090.front.sepia.ceph.com/smithi090/g' /etc/hosts 2024-04-18T10:25:50.665 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/172.21.15.90/172.21.15.90/g' /etc/hosts 2024-04-18T10:25:50.735 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi090.front.sepia.ceph.com/smithi090/g' /etc/hostname 2024-04-18T10:25:50.803 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090 2024-04-18T10:25:50.871 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-18T10:25:50.926 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-18T10:25:50.926 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-18T10:25:50.926 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-18T10:25:50.926 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T10:25:50.927 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T10:25:50.928 INFO:teuthology.provision.fog.smithi090:Deploy complete! 2024-04-18T10:25:50.928 DEBUG:teuthology.parallel:result is None 2024-04-18T10:25:53.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:25:56.606 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:26:02.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:26:05.662 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-18T10:26:11.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:26:11.895 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T10:26:11.895 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T10:26:27.482 INFO:teuthology.provision.fog.smithi159:Node is ready 2024-04-18T10:26:27.482 DEBUG:teuthology.orchestra.run.smithi159:> hostname 2024-04-18T10:26:27.500 INFO:teuthology.orchestra.run.smithi159.stdout:smithi046 2024-04-18T10:26:27.500 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi046 /etc/hosts 2024-04-18T10:26:27.557 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-04-18T10:26:27.557 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2024-04-18T10:26:27.704 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi046/smithi159/g' /etc/hosts 2024-04-18T10:26:27.735 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.46/172.21.15.159/g' /etc/hosts 2024-04-18T10:26:27.804 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi046/smithi159/g' /etc/hostname 2024-04-18T10:26:27.874 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2024-04-18T10:26:27.941 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-04-18T10:26:27.997 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-04-18T10:26:27.997 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-04-18T10:26:27.997 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-04-18T10:26:27.997 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-04-18T10:26:27.997 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-04-18T10:26:27.997 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-04-18T10:26:27.997 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T10:26:27.998 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-04-18T10:26:27.998 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-04-18T10:26:27.998 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T10:26:27.998 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-04-18T10:26:27.998 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T10:26:27.998 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T10:26:27.998 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T10:26:27.998 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2024-04-18T10:26:27.999 DEBUG:teuthology.parallel:result is None 2024-04-18T10:26:28.213 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com 2024-04-18T10:26:28.213 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T10:26:28.239 INFO:teuthology.lock.ops:Checking smithi118.front.sepia.ceph.com 2024-04-18T10:26:28.239 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T10:26:28.255 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2024-04-18T10:26:28.256 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T10:26:28.274 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server 2024-04-18T10:26:28.274 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-18T10:26:28.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:26:28.377 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-18T10:26:28.544 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-18T10:26:28.544 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-04-18T10:26:28.599 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-04-18T10:26:28.599 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-18T10:26:28.653 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T10:26:28.654 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-18T10:26:28.655 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T10:26:28.655 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T10:26:28.655 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T10:26:28.655 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-18T10:26:28.673 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: set os type and version on server 2024-04-18T10:26:28.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-04-18T10:26:28.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:26:28.761 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-04-18T10:26:28.937 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-04-18T10:26:28.938 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-04-18T10:26:28.993 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-04-18T10:26:28.993 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-04-18T10:26:29.049 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T10:26:29.050 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-04-18T10:26:29.050 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T10:26:29.050 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T10:26:29.050 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T10:26:29.050 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-04-18T10:26:29.073 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2024-04-18T10:26:29.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-18T10:26:29.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T10:26:29.159 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-04-18T10:26:29.225 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-18T10:26:29.225 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-04-18T10:26:29.279 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-04-18T10:26:29.280 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-04-18T10:26:29.334 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-04-18T10:26:29.334 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-04-18T10:26:29.334 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-18T10:26:29.335 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-18T10:26:29.336 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-04-18T10:26:29.356 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T10:26:29.453 INFO:teuthology.dispatcher.supervisor:Running job 7660982 2024-04-18T10:26:29.453 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-17_20:24:14-fs-main-distro-default-smithi/7660982 --name teuthology-2024-04-17_20:24:14-fs-main-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/iozone}} -- /home/teuthworker/archive/teuthology-2024-04-17_20:24:14-fs-main-distro-default-smithi/7660982/orig.config.yaml 2024-04-18T10:26:29.456 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-17_20:24:14-fs-main-distro-default-smithi/7660982 2024-04-18T10:26:29.456 INFO:teuthology.dispatcher.supervisor:Job PID: 3409804 2024-04-18T10:26:29.456 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-18T10:28:29.557 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T10:30:29.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T10:32:29.907 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T10:34:30.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T10:36:30.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T10:36:30.331 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1