2024-04-01T16:40:55.345 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-01T16:40:55.346 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T16:40:55.483 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f91772f5c10>, signals=[15]) 2024-04-01T16:40:55.483 INFO:teuthology.lock.ops:Start node 'smithi090.front.sepia.ceph.com' reimaging 2024-04-01T16:40:55.484 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: reset os type and version on server 2024-04-01T16:40:55.484 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-01T16:40:55.506 INFO:teuthology.lock.ops:Start node 'smithi142.front.sepia.ceph.com' reimaging 2024-04-01T16:40:55.506 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: reset os type and version on server 2024-04-01T16:40:55.506 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-04-01T16:40:55.531 INFO:teuthology.lock.ops:Start node 'smithi174.front.sepia.ceph.com' reimaging 2024-04-01T16:40:55.531 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: reset os type and version on server 2024-04-01T16:40:55.531 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-04-01T16:40:55.614 DEBUG:teuthology.provision.fog.smithi090:Requesting image smithi_centos_9.stream (ID 10) 2024-04-01T16:40:55.665 DEBUG:teuthology.provision.fog.smithi174:Requesting image smithi_centos_9.stream (ID 10) 2024-04-01T16:40:55.671 DEBUG:teuthology.provision.fog.smithi142:Requesting image smithi_centos_9.stream (ID 10) 2024-04-01T16:40:55.678 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of centos 9.stream 2024-04-01T16:40:55.709 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 9.stream 2024-04-01T16:40:55.717 INFO:teuthology.provision.fog.smithi142:Scheduling deploy of centos 9.stream 2024-04-01T16:40:55.871 INFO:teuthology.orchestra.console.smithi090:Power off 2024-04-01T16:40:55.871 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-01T16:40:55.893 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-04-01T16:40:55.894 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:40:55.916 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-01T16:40:55.934 INFO:teuthology.orchestra.console.smithi174:Power off 2024-04-01T16:40:55.934 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-01T16:40:55.947 INFO:teuthology.orchestra.console.smithi142:Power off 2024-04-01T16:40:55.947 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-01T16:40:55.955 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2024-04-01T16:40:55.956 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:40:55.968 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2024-04-01T16:40:55.969 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:40:55.977 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-04-01T16:40:55.990 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-04-01T16:40:59.921 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:40:59.978 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:40:59.990 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:00.031 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-01T16:41:00.086 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-04-01T16:41:00.099 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-04-01T16:41:04.036 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:04.087 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:04.100 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:04.145 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-01T16:41:04.197 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-04-01T16:41:04.207 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-04-01T16:41:04.298 INFO:teuthology.orchestra.console.smithi174:Power off completed 2024-04-01T16:41:04.398 INFO:teuthology.orchestra.console.smithi174:Power on 2024-04-01T16:41:04.399 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-01T16:41:04.420 DEBUG:teuthology.orchestra.console.smithi174:power on output: Chassis Power Control: Up/On 2024-04-01T16:41:04.420 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:04.440 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-04-01T16:41:08.149 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:08.208 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:08.258 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-04-01T16:41:08.316 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-04-01T16:41:08.360 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-04-01T16:41:08.417 INFO:teuthology.orchestra.console.smithi142:Power off completed 2024-04-01T16:41:08.441 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:08.461 INFO:teuthology.orchestra.console.smithi090:Power on 2024-04-01T16:41:08.461 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-01T16:41:08.483 DEBUG:teuthology.orchestra.console.smithi090:power on output: Chassis Power Control: Up/On 2024-04-01T16:41:08.484 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:08.506 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-04-01T16:41:08.518 INFO:teuthology.orchestra.console.smithi142:Power on 2024-04-01T16:41:08.518 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-01T16:41:08.544 DEBUG:teuthology.orchestra.console.smithi142:power on output: Chassis Power Control: Up/On 2024-04-01T16:41:08.544 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:08.552 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-04-01T16:41:08.564 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-04-01T16:41:12.510 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:12.553 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:12.565 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T16:41:12.620 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-04-01T16:41:12.662 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-04-01T16:41:12.673 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-04-01T16:41:12.720 INFO:teuthology.orchestra.console.smithi090:Power on completed 2024-04-01T16:41:12.762 INFO:teuthology.orchestra.console.smithi174:Power on completed 2024-04-01T16:41:12.774 INFO:teuthology.orchestra.console.smithi142:Power on completed 2024-04-01T16:41:12.822 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish 2024-04-01T16:41:12.863 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2024-04-01T16:41:12.875 INFO:teuthology.provision.fog.smithi142:Waiting for deploy to finish 2024-04-01T16:43:43.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:43:43.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:43:46.686 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:43:52.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:43:55.774 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:44:01.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:01.950 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-01T16:44:04.830 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:44:07.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:10.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:11.006 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-01T16:44:13.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:13.886 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:44:16.802 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-01T16:44:17.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:19.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:20.062 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-01T16:44:22.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:22.942 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:44:25.886 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-01T16:44:26.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:28.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:29.122 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-01T16:44:31.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:31.998 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:44:34.942 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-01T16:44:35.123 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:38.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:38.174 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-01T16:44:40.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:41.058 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:44:43.998 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-01T16:44:44.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:47.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:50.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:44:53.058 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-01T16:44:59.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:02.146 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-01T16:45:02.622 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-01T16:45:05.438 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:45:08.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:08.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:11.234 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-01T16:45:11.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:11.678 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-01T16:45:14.494 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:45:17.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:17.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:20.322 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-04-01T16:45:20.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:20.734 WARNING:teuthology.provision.fog.smithi142:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-01T16:45:23.550 WARNING:teuthology.provision.fog.smithi174:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-04-01T16:45:26.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:26.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:27.074 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-01T16:45:27.075 DEBUG:teuthology.orchestra.run.smithi142:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-01T16:45:29.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:45:29.938 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-01T16:45:29.938 DEBUG:teuthology.orchestra.run.smithi174:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-01T16:45:42.389 INFO:teuthology.provision.fog.smithi142:Node is ready 2024-04-01T16:45:42.389 DEBUG:teuthology.orchestra.run.smithi142:> hostname 2024-04-01T16:45:42.407 INFO:teuthology.orchestra.run.smithi142.stdout:smithi046 2024-04-01T16:45:42.407 DEBUG:teuthology.orchestra.run.smithi142:> grep smithi046 /etc/hosts 2024-04-01T16:45:42.466 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-04-01T16:45:42.466 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname smithi142 2024-04-01T16:45:42.615 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i -e 's/smithi046/smithi142/g' /etc/hosts 2024-04-01T16:45:42.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i -e 's/172.21.15.46/172.21.15.142/g' /etc/hosts 2024-04-01T16:45:42.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i -e 's/smithi046/smithi142/g' /etc/hostname 2024-04-01T16:45:42.790 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname smithi142 2024-04-01T16:45:42.861 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T16:45:42.920 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T16:45:42.921 INFO:teuthology.provision.fog.smithi142:Deploy complete! 2024-04-01T16:45:42.923 DEBUG:teuthology.parallel:result is None 2024-04-01T16:45:45.323 INFO:teuthology.provision.fog.smithi174:Node is ready 2024-04-01T16:45:45.323 DEBUG:teuthology.orchestra.run.smithi174:> hostname 2024-04-01T16:45:45.340 INFO:teuthology.orchestra.run.smithi174.stdout:smithi046 2024-04-01T16:45:45.341 DEBUG:teuthology.orchestra.run.smithi174:> grep smithi046 /etc/hosts 2024-04-01T16:45:45.399 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-04-01T16:45:45.400 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2024-04-01T16:45:45.544 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi046/smithi174/g' /etc/hosts 2024-04-01T16:45:45.576 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/172.21.15.46/172.21.15.174/g' /etc/hosts 2024-04-01T16:45:45.645 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi046/smithi174/g' /etc/hostname 2024-04-01T16:45:45.715 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2024-04-01T16:45:45.782 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-04-01T16:45:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-04-01T16:45:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-04-01T16:45:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-04-01T16:45:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-04-01T16:45:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-04-01T16:45:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-04-01T16:45:45.838 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T16:45:45.839 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-04-01T16:45:45.839 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-04-01T16:45:45.839 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T16:45:45.839 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-04-01T16:45:45.839 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T16:45:45.839 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T16:45:45.839 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T16:45:45.839 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2024-04-01T16:45:45.840 DEBUG:teuthology.parallel:result is None 2024-04-01T16:46:26.369 WARNING:teuthology.provision.fog.smithi090:timed out 2024-04-01T16:46:32.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:46:32.698 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-01T16:46:32.699 DEBUG:teuthology.orchestra.run.smithi090:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-01T16:46:33.059 INFO:teuthology.provision.fog.smithi090:Node is ready 2024-04-01T16:46:33.060 DEBUG:teuthology.orchestra.run.smithi090:> hostname 2024-04-01T16:46:33.074 INFO:teuthology.orchestra.run.smithi090.stdout:smithi046 2024-04-01T16:46:33.074 DEBUG:teuthology.orchestra.run.smithi090:> grep smithi046 /etc/hosts 2024-04-01T16:46:33.129 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-04-01T16:46:33.129 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090 2024-04-01T16:46:33.269 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi046/smithi090/g' /etc/hosts 2024-04-01T16:46:33.298 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/172.21.15.46/172.21.15.90/g' /etc/hosts 2024-04-01T16:46:33.365 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi046/smithi090/g' /etc/hostname 2024-04-01T16:46:33.433 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090 2024-04-01T16:46:33.505 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-01T16:46:33.559 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T16:46:33.560 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T16:46:33.561 INFO:teuthology.provision.fog.smithi090:Deploy complete! 2024-04-01T16:46:33.561 DEBUG:teuthology.parallel:result is None 2024-04-01T16:46:33.777 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com 2024-04-01T16:46:33.777 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-01T16:46:33.798 INFO:teuthology.lock.ops:Checking smithi142.front.sepia.ceph.com 2024-04-01T16:46:33.798 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-01T16:46:33.823 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2024-04-01T16:46:33.823 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-01T16:46:33.849 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server 2024-04-01T16:46:33.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-01T16:46:33.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:46:33.936 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-01T16:46:34.000 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-01T16:46:34.000 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-04-01T16:46:34.054 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-04-01T16:46:34.055 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-01T16:46:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-01T16:46:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-01T16:46:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-01T16:46:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-01T16:46:34.109 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T16:46:34.110 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T16:46:34.111 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-01T16:46:34.132 INFO:teuthology.lock.ops:Updating [smithi142.front.sepia.ceph.com]: set os type and version on server 2024-04-01T16:46:34.132 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-01T16:46:34.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:46:34.222 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-04-01T16:46:34.392 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-01T16:46:34.393 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-04-01T16:46:34.448 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-04-01T16:46:34.448 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-04-01T16:46:34.502 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-04-01T16:46:34.502 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-04-01T16:46:34.502 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-04-01T16:46:34.502 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-04-01T16:46:34.502 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-04-01T16:46:34.502 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-04-01T16:46:34.503 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T16:46:34.503 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-04-01T16:46:34.503 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-04-01T16:46:34.503 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T16:46:34.503 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-04-01T16:46:34.503 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T16:46:34.503 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T16:46:34.503 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T16:46:34.503 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-04-01T16:46:34.531 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2024-04-01T16:46:34.531 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-04-01T16:46:34.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T16:46:34.620 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-04-01T16:46:34.793 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-04-01T16:46:34.793 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-04-01T16:46:34.847 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-04-01T16:46:34.847 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-04-01T16:46:34.902 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T16:46:34.903 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-04-01T16:46:34.903 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T16:46:34.903 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T16:46:34.903 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T16:46:34.903 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-04-01T16:46:34.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T16:46:35.020 INFO:teuthology.dispatcher.supervisor:Running job 7633991 2024-04-01T16:46:35.021 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-04-01_14:11:32-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7633991 --name vshankar-2024-04-01_14:11:32-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/kernel_untar_build}} -- /home/teuthworker/archive/vshankar-2024-04-01_14:11:32-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7633991/orig.config.yaml 2024-04-01T16:46:35.024 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-04-01_14:11:32-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7633991 2024-04-01T16:46:35.024 INFO:teuthology.dispatcher.supervisor:Job PID: 3468499 2024-04-01T16:46:35.024 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-01T16:48:35.121 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T16:50:35.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T16:52:35.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T16:54:35.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T16:56:35.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T16:58:36.012 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T17:00:36.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T17:00:36.254 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1