2024-04-01T10:03:04.947 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-01T10:03:04.947 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:03:05.134 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f33e2a75c10>, signals=[15]) 2024-04-01T10:03:05.134 INFO:teuthology.lock.ops:Start node 'smithi042.front.sepia.ceph.com' reimaging 2024-04-01T10:03:05.134 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: reset os type and version on server 2024-04-01T10:03:05.134 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-04-01T10:03:05.162 INFO:teuthology.lock.ops:Start node 'smithi069.front.sepia.ceph.com' reimaging 2024-04-01T10:03:05.162 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: reset os type and version on server 2024-04-01T10:03:05.162 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-04-01T10:03:05.187 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging 2024-04-01T10:03:05.187 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server 2024-04-01T10:03:05.188 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-01T10:03:05.279 DEBUG:teuthology.provision.fog.smithi042:Requesting image smithi_centos_9.stream (ID 10) 2024-04-01T10:03:05.305 DEBUG:teuthology.provision.fog.smithi069:Requesting image smithi_centos_9.stream (ID 10) 2024-04-01T10:03:05.313 DEBUG:teuthology.provision.fog.smithi169:Requesting image smithi_centos_9.stream (ID 10) 2024-04-01T10:03:05.332 INFO:teuthology.provision.fog.smithi042:Scheduling deploy of centos 9.stream 2024-04-01T10:03:05.346 INFO:teuthology.provision.fog.smithi069:Scheduling deploy of centos 9.stream 2024-04-01T10:03:05.352 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of centos 9.stream 2024-04-01T10:03:05.571 INFO:teuthology.orchestra.console.smithi069:Power off 2024-04-01T10:03:05.572 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-01T10:03:05.585 INFO:teuthology.orchestra.console.smithi042:Power off 2024-04-01T10:03:05.585 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-01T10:03:05.597 INFO:teuthology.orchestra.console.smithi169:Power off 2024-04-01T10:03:05.597 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-01T10:03:05.606 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2024-04-01T10:03:05.606 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:05.614 DEBUG:teuthology.orchestra.console.smithi042:power off output: Chassis Power Control: Down/Off 2024-04-01T10:03:05.614 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:05.623 DEBUG:teuthology.orchestra.console.smithi169:power off output: Chassis Power Control: Down/Off 2024-04-01T10:03:05.623 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:05.632 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-04-01T10:03:05.637 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-04-01T10:03:05.650 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-01T10:03:09.634 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:09.641 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:09.650 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:09.743 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-04-01T10:03:09.750 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-04-01T10:03:09.759 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-01T10:03:13.746 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:13.754 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:13.761 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:13.856 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-04-01T10:03:13.863 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-04-01T10:03:13.872 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-01T10:03:13.958 INFO:teuthology.orchestra.console.smithi069:Power off completed 2024-04-01T10:03:14.058 INFO:teuthology.orchestra.console.smithi069:Power on 2024-04-01T10:03:14.059 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-01T10:03:14.081 DEBUG:teuthology.orchestra.console.smithi069:power on output: Chassis Power Control: Up/On 2024-04-01T10:03:14.081 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:14.105 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-04-01T10:03:17.866 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:17.878 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:17.981 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-04-01T10:03:17.990 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2024-04-01T10:03:18.083 INFO:teuthology.orchestra.console.smithi042:Power off completed 2024-04-01T10:03:18.091 INFO:teuthology.orchestra.console.smithi169:Power off completed 2024-04-01T10:03:18.106 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:18.185 INFO:teuthology.orchestra.console.smithi042:Power on 2024-04-01T10:03:18.185 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-01T10:03:18.193 INFO:teuthology.orchestra.console.smithi169:Power on 2024-04-01T10:03:18.193 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-01T10:03:18.207 DEBUG:teuthology.orchestra.console.smithi042:power on output: Chassis Power Control: Up/On 2024-04-01T10:03:18.208 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:18.215 DEBUG:teuthology.orchestra.console.smithi169:power on output: Chassis Power Control: Up/On 2024-04-01T10:03:18.216 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:18.223 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-04-01T10:03:18.230 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-04-01T10:03:18.237 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2024-04-01T10:03:22.226 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:22.234 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:22.241 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:22.343 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2024-04-01T10:03:22.350 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is off 2024-04-01T10:03:22.400 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-04-01T10:03:22.500 INFO:teuthology.orchestra.console.smithi069:Power on completed 2024-04-01T10:03:22.601 INFO:teuthology.provision.fog.smithi069:Waiting for deploy to finish 2024-04-01T10:03:26.346 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:26.353 DEBUG:teuthology.orchestra.console.smithi169:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-01T10:03:26.455 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2024-04-01T10:03:26.462 DEBUG:teuthology.orchestra.console.smithi169:check power output: Chassis Power is on 2024-04-01T10:03:26.557 INFO:teuthology.orchestra.console.smithi042:Power on completed 2024-04-01T10:03:26.563 INFO:teuthology.orchestra.console.smithi169:Power on completed 2024-04-01T10:03:26.658 INFO:teuthology.provision.fog.smithi042:Waiting for deploy to finish 2024-04-01T10:03:26.664 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish 2024-04-01T10:05:42.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:05:45.374 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-01T10:05:51.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:05:53.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:05:54.434 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-01T10:05:56.350 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:06:00.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:02.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:03.518 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-01T10:06:05.406 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:06:09.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:11.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:12.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:12.574 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-01T10:06:14.466 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:06:15.486 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-01T10:06:18.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:20.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:21.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:21.630 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-01T10:06:23.550 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:06:24.546 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-01T10:06:27.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:29.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:30.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:30.686 WARNING:teuthology.provision.fog.smithi042:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-01T10:06:32.606 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:06:33.630 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-01T10:06:36.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:38.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:39.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:41.662 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:06:42.686 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-01T10:06:47.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:48.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:50.750 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:06:51.742 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-01T10:06:56.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:06:57.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:07:00.798 WARNING:teuthology.provision.fog.smithi169:[Errno None] Unable to connect to port 22 on 172.21.15.169 2024-04-01T10:07:06.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:07:15.230 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:07:21.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:07:24.286 WARNING:teuthology.provision.fog.smithi069:[Errno None] Unable to connect to port 22 on 172.21.15.69 2024-04-01T10:07:30.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:07:30.663 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-01T10:07:30.664 DEBUG:teuthology.orchestra.run.smithi069:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-01T10:07:36.690 WARNING:teuthology.provision.fog.smithi042:timed out 2024-04-01T10:07:42.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:07:42.991 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-01T10:07:42.991 DEBUG:teuthology.orchestra.run.smithi042:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-01T10:07:43.316 INFO:teuthology.provision.fog.smithi042:Node is ready 2024-04-01T10:07:43.317 DEBUG:teuthology.orchestra.run.smithi042:> hostname 2024-04-01T10:07:43.331 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042.front.sepia.ceph.com 2024-04-01T10:07:43.332 DEBUG:teuthology.orchestra.run.smithi042:> grep smithi042.front.sepia.ceph.com /etc/hosts 2024-04-01T10:07:43.387 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42 smithi042.front.sepia.ceph.com smithi042 2024-04-01T10:07:43.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname smithi042 2024-04-01T10:07:43.496 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/smithi042.front.sepia.ceph.com/smithi042/g' /etc/hosts 2024-04-01T10:07:43.528 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/172.21.15.42/172.21.15.42/g' /etc/hosts 2024-04-01T10:07:43.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i -e 's/smithi042.front.sepia.ceph.com/smithi042/g' /etc/hostname 2024-04-01T10:07:43.670 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname smithi042 2024-04-01T10:07:43.741 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-04-01T10:07:43.795 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T10:07:43.796 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T10:07:43.797 INFO:teuthology.provision.fog.smithi042:Deploy complete! 2024-04-01T10:07:43.798 DEBUG:teuthology.parallel:result is None 2024-04-01T10:07:51.455 INFO:teuthology.provision.fog.smithi069:Node is ready 2024-04-01T10:07:51.456 DEBUG:teuthology.orchestra.run.smithi069:> hostname 2024-04-01T10:07:51.475 INFO:teuthology.orchestra.run.smithi069.stdout:smithi046 2024-04-01T10:07:51.475 DEBUG:teuthology.orchestra.run.smithi069:> grep smithi046 /etc/hosts 2024-04-01T10:07:51.532 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-04-01T10:07:51.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname smithi069 2024-04-01T10:07:51.698 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi046/smithi069/g' /etc/hosts 2024-04-01T10:07:51.729 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/172.21.15.46/172.21.15.69/g' /etc/hosts 2024-04-01T10:07:51.799 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i -e 's/smithi046/smithi069/g' /etc/hostname 2024-04-01T10:07:51.869 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname smithi069 2024-04-01T10:07:51.936 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-04-01T10:07:51.991 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-04-01T10:07:51.991 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-04-01T10:07:51.991 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-04-01T10:07:51.991 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-04-01T10:07:51.991 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T10:07:51.992 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T10:07:51.992 INFO:teuthology.provision.fog.smithi069:Deploy complete! 2024-04-01T10:07:51.993 DEBUG:teuthology.parallel:result is None 2024-04-01T10:08:06.819 WARNING:teuthology.provision.fog.smithi169:timed out 2024-04-01T10:08:12.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:08:13.048 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-01T10:08:13.048 DEBUG:teuthology.orchestra.run.smithi169:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-01T10:08:13.379 INFO:teuthology.provision.fog.smithi169:Node is ready 2024-04-01T10:08:13.379 DEBUG:teuthology.orchestra.run.smithi169:> hostname 2024-04-01T10:08:13.393 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com 2024-04-01T10:08:13.393 DEBUG:teuthology.orchestra.run.smithi169:> grep smithi169.front.sepia.ceph.com /etc/hosts 2024-04-01T10:08:13.448 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169 2024-04-01T10:08:13.448 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169 2024-04-01T10:08:13.566 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hosts 2024-04-01T10:08:13.594 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/172.21.15.169/172.21.15.169/g' /etc/hosts 2024-04-01T10:08:13.661 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hostname 2024-04-01T10:08:13.728 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169 2024-04-01T10:08:13.794 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-01T10:08:13.848 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T10:08:13.849 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T10:08:13.850 INFO:teuthology.provision.fog.smithi169:Deploy complete! 2024-04-01T10:08:13.850 DEBUG:teuthology.parallel:result is None 2024-04-01T10:08:14.062 INFO:teuthology.lock.ops:Checking smithi042.front.sepia.ceph.com 2024-04-01T10:08:14.062 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-01T10:08:14.087 INFO:teuthology.lock.ops:Checking smithi069.front.sepia.ceph.com 2024-04-01T10:08:14.087 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-01T10:08:14.104 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com 2024-04-01T10:08:14.104 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-01T10:08:14.121 INFO:teuthology.lock.ops:Updating [smithi042.front.sepia.ceph.com]: set os type and version on server 2024-04-01T10:08:14.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-01T10:08:14.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:08:14.226 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-04-01T10:08:14.409 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-01T10:08:14.409 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-04-01T10:08:14.463 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-04-01T10:08:14.464 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T10:08:14.519 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T10:08:14.520 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T10:08:14.520 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-04-01T10:08:14.537 INFO:teuthology.lock.ops:Updating [smithi069.front.sepia.ceph.com]: set os type and version on server 2024-04-01T10:08:14.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-01T10:08:14.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:08:14.626 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-04-01T10:08:14.796 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-04-01T10:08:14.797 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-04-01T10:08:14.850 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-04-01T10:08:14.851 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-04-01T10:08:14.904 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-04-01T10:08:14.904 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-04-01T10:08:14.904 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T10:08:14.905 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T10:08:14.906 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-04-01T10:08:14.929 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server 2024-04-01T10:08:14.929 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-01T10:08:14.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-01T10:08:15.014 DEBUG:teuthology.orchestra.run.smithi169:> true 2024-04-01T10:08:15.081 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com' 2024-04-01T10:08:15.081 DEBUG:teuthology.orchestra.run.smithi169:> uname -m 2024-04-01T10:08:15.136 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64 2024-04-01T10:08:15.136 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release 2024-04-01T10:08:15.191 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-01T10:08:15.192 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-01T10:08:15.193 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server 2024-04-01T10:08:15.213 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:08:15.300 INFO:teuthology.dispatcher.supervisor:Running job 7626032 2024-04-01T10:08:15.300 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_pdiazbou@teuthology --archive /home/teuthworker/archive/pdiazbou-2024-03-27_11:08:38-rados-wip-pereman2-blob_use_tracker_t-power-of-two-distro-default-smithi/7626032 --name pdiazbou-2024-03-27_11:08:38-rados-wip-pereman2-blob_use_tracker_t-power-of-two-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} -- /home/teuthworker/archive/pdiazbou-2024-03-27_11:08:38-rados-wip-pereman2-blob_use_tracker_t-power-of-two-distro-default-smithi/7626032/orig.config.yaml 2024-04-01T10:08:15.303 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdiazbou-2024-03-27_11:08:38-rados-wip-pereman2-blob_use_tracker_t-power-of-two-distro-default-smithi/7626032 2024-04-01T10:08:15.304 INFO:teuthology.dispatcher.supervisor:Job PID: 1820167 2024-04-01T10:08:15.304 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-01T10:10:15.392 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:12:15.546 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:14:15.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:16:15.880 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:18:16.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:20:16.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:22:16.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:24:16.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:26:16.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:28:16.919 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:30:17.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-01T10:30:17.185 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1