2024-03-09T17:35:32.247 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-09T17:35:32.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T17:35:32.434 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffaedccac10>, signals=[15]) 2024-03-09T17:35:32.434 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging 2024-03-09T17:35:32.434 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server 2024-03-09T17:35:32.435 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-03-09T17:35:32.454 INFO:teuthology.lock.ops:Start node 'smithi019.front.sepia.ceph.com' reimaging 2024-03-09T17:35:32.455 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: reset os type and version on server 2024-03-09T17:35:32.455 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-03-09T17:35:32.479 INFO:teuthology.lock.ops:Start node 'smithi145.front.sepia.ceph.com' reimaging 2024-03-09T17:35:32.479 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: reset os type and version on server 2024-03-09T17:35:32.479 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-03-09T17:35:32.580 DEBUG:teuthology.provision.fog.smithi019:Requesting image smithi_centos_8.stream (ID 3) 2024-03-09T17:35:32.583 DEBUG:teuthology.provision.fog.smithi002:Requesting image smithi_centos_8.stream (ID 3) 2024-03-09T17:35:32.613 DEBUG:teuthology.provision.fog.smithi145:Requesting image smithi_centos_8.stream (ID 3) 2024-03-09T17:35:32.631 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 8.stream 2024-03-09T17:35:32.637 INFO:teuthology.provision.fog.smithi019:Scheduling deploy of centos 8.stream 2024-03-09T17:35:32.663 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of centos 8.stream 2024-03-09T17:35:32.914 INFO:teuthology.orchestra.console.smithi019:Power off 2024-03-09T17:35:32.915 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-09T17:35:32.929 INFO:teuthology.orchestra.console.smithi002:Power off 2024-03-09T17:35:32.930 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-09T17:35:32.943 INFO:teuthology.orchestra.console.smithi145:Power off 2024-03-09T17:35:32.943 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-09T17:35:32.950 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-03-09T17:35:32.950 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:32.960 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2024-03-09T17:35:32.960 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:32.968 DEBUG:teuthology.orchestra.console.smithi145:power off output: Chassis Power Control: Down/Off 2024-03-09T17:35:32.968 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:32.976 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-09T17:35:32.982 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-03-09T17:35:32.989 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-03-09T17:35:36.981 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:36.989 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:36.995 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:37.091 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-09T17:35:37.097 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-03-09T17:35:37.102 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-03-09T17:35:41.095 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:41.102 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:41.107 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:41.204 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-09T17:35:41.209 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-03-09T17:35:41.217 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-03-09T17:35:41.310 INFO:teuthology.orchestra.console.smithi002:Power off completed 2024-03-09T17:35:41.411 INFO:teuthology.orchestra.console.smithi002:Power on 2024-03-09T17:35:41.411 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-09T17:35:41.436 DEBUG:teuthology.orchestra.console.smithi002:power on output: Chassis Power Control: Up/On 2024-03-09T17:35:41.436 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:41.457 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-03-09T17:35:45.206 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:45.218 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:45.314 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-03-09T17:35:45.326 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is off 2024-03-09T17:35:45.416 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-03-09T17:35:45.427 INFO:teuthology.orchestra.console.smithi145:Power off completed 2024-03-09T17:35:45.458 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:45.516 INFO:teuthology.orchestra.console.smithi019:Power on 2024-03-09T17:35:45.517 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-09T17:35:45.529 INFO:teuthology.orchestra.console.smithi145:Power on 2024-03-09T17:35:45.529 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-09T17:35:45.538 DEBUG:teuthology.orchestra.console.smithi019:power on output: Chassis Power Control: Up/On 2024-03-09T17:35:45.538 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:45.550 DEBUG:teuthology.orchestra.console.smithi145:power on output: Chassis Power Control: Up/On 2024-03-09T17:35:45.551 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:45.562 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-03-09T17:35:45.566 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2024-03-09T17:35:45.573 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is off 2024-03-09T17:35:49.567 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:49.573 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:49.580 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:49.676 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-03-09T17:35:49.682 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2024-03-09T17:35:49.687 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-03-09T17:35:49.783 INFO:teuthology.orchestra.console.smithi002:Power on completed 2024-03-09T17:35:49.788 INFO:teuthology.orchestra.console.smithi145:Power on completed 2024-03-09T17:35:49.884 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2024-03-09T17:35:49.889 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2024-03-09T17:35:53.678 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-09T17:35:53.787 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-09T17:35:53.888 INFO:teuthology.orchestra.console.smithi019:Power on completed 2024-03-09T17:35:53.988 INFO:teuthology.provision.fog.smithi019:Waiting for deploy to finish 2024-03-09T17:38:20.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:23.970 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-03-09T17:38:24.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:27.970 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-09T17:38:29.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:33.054 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-03-09T17:38:33.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:35.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:37.058 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-09T17:38:39.038 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-03-09T17:38:39.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:42.110 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-03-09T17:38:43.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:45.041 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:46.114 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-09T17:38:48.098 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-03-09T17:38:48.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:51.166 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-03-09T17:38:52.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:54.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:55.166 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-09T17:38:57.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:38:57.186 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-03-09T17:39:00.222 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-03-09T17:39:01.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:03.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:04.222 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-09T17:39:06.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:06.274 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-03-09T17:39:09.282 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-03-09T17:39:10.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:12.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:13.282 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-09T17:39:15.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:15.362 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-03-09T17:39:19.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:21.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:24.446 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-03-09T17:39:30.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:48.898 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-03-09T17:39:54.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:39:57.982 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-03-09T17:40:03.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:40:05.250 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-09T17:40:05.250 DEBUG:teuthology.orchestra.run.smithi145:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-09T17:40:15.288 WARNING:teuthology.provision.fog.smithi002:timed out 2024-03-09T17:40:19.289 WARNING:teuthology.provision.fog.smithi019:timed out 2024-03-09T17:40:21.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:40:21.551 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-09T17:40:21.552 DEBUG:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-09T17:40:21.802 INFO:teuthology.provision.fog.smithi002:Node is ready 2024-03-09T17:40:21.803 DEBUG:teuthology.orchestra.run.smithi002:> hostname 2024-03-09T17:40:21.822 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2024-03-09T17:40:21.822 DEBUG:teuthology.orchestra.run.smithi002:> grep smithi002.front.sepia.ceph.com /etc/hosts 2024-03-09T17:40:21.882 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2024-03-09T17:40:21.882 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2024-03-09T17:40:21.989 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hosts 2024-03-09T17:40:22.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/172.21.15.2/172.21.15.2/g' /etc/hosts 2024-03-09T17:40:22.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hostname 2024-03-09T17:40:22.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2024-03-09T17:40:22.231 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-03-09T17:40:22.290 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-09T17:40:22.291 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-09T17:40:22.292 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2024-03-09T17:40:22.293 DEBUG:teuthology.parallel:result is None 2024-03-09T17:40:25.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:40:25.504 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-09T17:40:25.504 DEBUG:teuthology.orchestra.run.smithi019:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-09T17:40:25.751 INFO:teuthology.provision.fog.smithi019:Node is ready 2024-03-09T17:40:25.751 DEBUG:teuthology.orchestra.run.smithi019:> hostname 2024-03-09T17:40:25.771 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019.front.sepia.ceph.com 2024-03-09T17:40:25.771 DEBUG:teuthology.orchestra.run.smithi019:> grep smithi019.front.sepia.ceph.com /etc/hosts 2024-03-09T17:40:25.832 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19 smithi019.front.sepia.ceph.com smithi019 2024-03-09T17:40:25.832 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname smithi019 2024-03-09T17:40:25.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/smithi019.front.sepia.ceph.com/smithi019/g' /etc/hosts 2024-03-09T17:40:25.979 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/172.21.15.19/172.21.15.19/g' /etc/hosts 2024-03-09T17:40:26.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/smithi019.front.sepia.ceph.com/smithi019/g' /etc/hostname 2024-03-09T17:40:26.119 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname smithi019 2024-03-09T17:40:26.188 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-03-09T17:40:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-03-09T17:40:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="8" 2024-03-09T17:40:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-03-09T17:40:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-03-09T17:40:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="8" 2024-03-09T17:40:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el8" 2024-03-09T17:40:26.247 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-09T17:40:26.248 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-03-09T17:40:26.248 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-09T17:40:26.248 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-03-09T17:40:26.248 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-09T17:40:26.248 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-09T17:40:26.248 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-09T17:40:26.248 INFO:teuthology.provision.fog.smithi019:Deploy complete! 2024-03-09T17:40:26.249 DEBUG:teuthology.parallel:result is None 2024-03-09T17:40:30.263 INFO:teuthology.provision.fog.smithi145:Node is ready 2024-03-09T17:40:30.263 DEBUG:teuthology.orchestra.run.smithi145:> hostname 2024-03-09T17:40:30.287 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2024-03-09T17:40:30.288 DEBUG:teuthology.orchestra.run.smithi145:> grep smithi145.front.sepia.ceph.com /etc/hosts 2024-03-09T17:40:30.349 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2024-03-09T17:40:30.349 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname smithi145 2024-03-09T17:40:30.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/smithi145.front.sepia.ceph.com/smithi145/g' /etc/hosts 2024-03-09T17:40:30.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/172.21.15.145/172.21.15.145/g' /etc/hosts 2024-03-09T17:40:30.587 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/smithi145.front.sepia.ceph.com/smithi145/g' /etc/hostname 2024-03-09T17:40:30.657 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname smithi145 2024-03-09T17:40:30.726 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-03-09T17:40:30.784 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-03-09T17:40:30.784 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8" 2024-03-09T17:40:30.784 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-03-09T17:40:30.784 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-09T17:40:30.785 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-09T17:40:30.786 INFO:teuthology.provision.fog.smithi145:Deploy complete! 2024-03-09T17:40:30.786 DEBUG:teuthology.parallel:result is None 2024-03-09T17:40:30.989 INFO:teuthology.lock.ops:Checking smithi002.front.sepia.ceph.com 2024-03-09T17:40:30.989 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-09T17:40:31.016 INFO:teuthology.lock.ops:Checking smithi019.front.sepia.ceph.com 2024-03-09T17:40:31.016 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-09T17:40:31.033 INFO:teuthology.lock.ops:Checking smithi145.front.sepia.ceph.com 2024-03-09T17:40:31.033 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-09T17:40:31.059 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: set os type and version on server 2024-03-09T17:40:31.059 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-09T17:40:31.060 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:40:31.135 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-03-09T17:40:31.213 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-03-09T17:40:31.213 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-03-09T17:40:31.275 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-03-09T17:40:31.275 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-03-09T17:40:31.337 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-03-09T17:40:31.337 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8" 2024-03-09T17:40:31.337 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-09T17:40:31.338 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-09T17:40:31.339 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-03-09T17:40:31.358 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: set os type and version on server 2024-03-09T17:40:31.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-09T17:40:31.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:40:31.433 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-03-09T17:40:31.512 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-09T17:40:31.512 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-03-09T17:40:31.574 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-03-09T17:40:31.574 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-03-09T17:40:31.636 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="8" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="8" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el8" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-09T17:40:31.637 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-09T17:40:31.638 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-03-09T17:40:31.667 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: set os type and version on server 2024-03-09T17:40:31.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-03-09T17:40:31.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-09T17:40:31.744 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-03-09T17:40:31.819 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-03-09T17:40:31.819 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-03-09T17:40:31.879 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-03-09T17:40:31.879 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-03-09T17:40:31.940 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-03-09T17:40:31.940 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8" 2024-03-09T17:40:31.940 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-03-09T17:40:31.940 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-03-09T17:40:31.940 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8" 2024-03-09T17:40:31.940 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2024-03-09T17:40:31.940 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 8" 2024-03-09T17:40:31.940 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-03-09T17:40:31.941 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-03-09T17:40:31.941 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-03-09T17:40:31.941 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-09T17:40:31.941 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-09T17:40:31.941 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-09T17:40:31.941 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-03-09T17:40:31.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T17:40:32.060 INFO:teuthology.dispatcher.supervisor:Running job 7587407 2024-03-09T17:40:32.061 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-08_16:20:46-rados-wip-yuri4-testing-2024-03-05-0854-distro-default-smithi/7587407 --name yuriw-2024-03-08_16:20:46-rados-wip-yuri4-testing-2024-03-05-0854-distro-default-smithi --description rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/reef backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/cache-snaps} -- /home/teuthworker/archive/yuriw-2024-03-08_16:20:46-rados-wip-yuri4-testing-2024-03-05-0854-distro-default-smithi/7587407/orig.config.yaml 2024-03-09T17:40:32.065 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-03-08_16:20:46-rados-wip-yuri4-testing-2024-03-05-0854-distro-default-smithi/7587407 2024-03-09T17:40:32.065 INFO:teuthology.dispatcher.supervisor:Job PID: 3523410 2024-03-09T17:40:32.065 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-09T17:42:32.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T17:44:32.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T17:46:32.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T17:48:32.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T17:50:32.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T17:52:33.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-09T17:52:33.080 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1