2024-01-12T00:13:18.098 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-12T00:13:18.098 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:13:18.271 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4d528bfc10>, signals=[15]) 2024-01-12T00:13:18.271 INFO:teuthology.lock.ops:Start node 'smithi107.front.sepia.ceph.com' reimaging 2024-01-12T00:13:18.271 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: reset os type and version on server 2024-01-12T00:13:18.271 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-01-12T00:13:18.292 INFO:teuthology.lock.ops:Start node 'smithi124.front.sepia.ceph.com' reimaging 2024-01-12T00:13:18.292 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: reset os type and version on server 2024-01-12T00:13:18.292 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2024-01-12T00:13:18.317 INFO:teuthology.lock.ops:Start node 'smithi188.front.sepia.ceph.com' reimaging 2024-01-12T00:13:18.317 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: reset os type and version on server 2024-01-12T00:13:18.317 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-01-12T00:13:18.409 DEBUG:teuthology.provision.fog.smithi107:Requesting image smithi_centos_9.stream (ID 10) 2024-01-12T00:13:18.448 DEBUG:teuthology.provision.fog.smithi188:Requesting image smithi_centos_9.stream (ID 10) 2024-01-12T00:13:18.461 INFO:teuthology.provision.fog.smithi107:Scheduling deploy of centos 9.stream 2024-01-12T00:13:18.493 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of centos 9.stream 2024-01-12T00:13:18.537 DEBUG:teuthology.provision.fog.smithi124:Requesting image smithi_centos_9.stream (ID 10) 2024-01-12T00:13:18.583 INFO:teuthology.provision.fog.smithi124:Scheduling deploy of centos 9.stream 2024-01-12T00:13:18.674 INFO:teuthology.orchestra.console.smithi107:Power off 2024-01-12T00:13:18.674 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-12T00:13:18.696 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-01-12T00:13:18.696 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:18.708 INFO:teuthology.orchestra.console.smithi188:Power off 2024-01-12T00:13:18.708 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-12T00:13:18.718 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-01-12T00:13:18.728 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-01-12T00:13:18.729 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:18.750 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-01-12T00:13:18.824 INFO:teuthology.orchestra.console.smithi124:Power off 2024-01-12T00:13:18.825 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-12T00:13:18.848 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2024-01-12T00:13:18.848 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:18.855 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-01-12T00:13:18.870 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2024-01-12T00:13:18.956 INFO:teuthology.orchestra.console.smithi188:Power on 2024-01-12T00:13:18.956 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-12T00:13:18.971 INFO:teuthology.orchestra.console.smithi124:Power off completed 2024-01-12T00:13:18.977 DEBUG:teuthology.orchestra.console.smithi188:power on output: Chassis Power Control: Up/On 2024-01-12T00:13:18.978 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:18.997 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-01-12T00:13:19.072 INFO:teuthology.orchestra.console.smithi124:Power on 2024-01-12T00:13:19.072 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-12T00:13:19.093 DEBUG:teuthology.orchestra.console.smithi124:power on output: Chassis Power Control: Up/On 2024-01-12T00:13:19.093 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:19.114 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2024-01-12T00:13:22.723 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:22.832 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-01-12T00:13:22.998 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:23.107 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-01-12T00:13:23.115 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:23.224 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2024-01-12T00:13:26.837 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:26.948 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-01-12T00:13:27.049 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-01-12T00:13:27.108 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:27.149 INFO:teuthology.orchestra.console.smithi107:Power on 2024-01-12T00:13:27.150 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-12T00:13:27.171 DEBUG:teuthology.orchestra.console.smithi107:power on output: Chassis Power Control: Up/On 2024-01-12T00:13:27.171 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:27.192 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-01-12T00:13:27.218 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-01-12T00:13:27.225 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:27.319 INFO:teuthology.orchestra.console.smithi188:Power on completed 2024-01-12T00:13:27.334 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2024-01-12T00:13:27.420 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish 2024-01-12T00:13:27.436 INFO:teuthology.orchestra.console.smithi124:Power on completed 2024-01-12T00:13:27.537 INFO:teuthology.provision.fog.smithi124:Waiting for deploy to finish 2024-01-12T00:13:31.197 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:31.305 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-01-12T00:13:35.310 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:13:35.482 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-01-12T00:13:35.582 INFO:teuthology.orchestra.console.smithi107:Power on completed 2024-01-12T00:13:35.683 INFO:teuthology.provision.fog.smithi107:Waiting for deploy to finish 2024-01-12T00:15:58.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:15:58.192 ERROR:teuthology.orchestra.connection:Error authenticating with smithi188.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-12T00:16:06.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:16:06.452 ERROR:teuthology.orchestra.connection:Error authenticating with smithi107.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-12T00:16:13.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:16:16.284 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2024-01-12T00:16:22.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:16:25.340 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2024-01-12T00:16:31.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:16:34.396 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2024-01-12T00:16:40.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:16:43.456 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2024-01-12T00:16:49.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:16:52.508 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2024-01-12T00:16:58.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:17:01.564 WARNING:teuthology.provision.fog.smithi124:[Errno None] Unable to connect to port 22 on 172.21.15.124 2024-01-12T00:17:02.194 WARNING:teuthology.provision.fog.smithi188:timed out 2024-01-12T00:17:07.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:17:08.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:17:10.455 WARNING:teuthology.provision.fog.smithi107:timed out 2024-01-12T00:17:16.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:17:34.784 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-01-12T00:17:40.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:17:43.836 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-01-12T00:17:49.838 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:17:52.896 WARNING:teuthology.provision.fog.smithi107:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-01-12T00:17:58.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:17:59.138 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-12T00:17:59.139 DEBUG:teuthology.orchestra.run.smithi107:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-12T00:18:07.568 WARNING:teuthology.provision.fog.smithi124:timed out 2024-01-12T00:18:08.197 WARNING:teuthology.provision.fog.smithi188:timed out 2024-01-12T00:18:13.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:18:13.860 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-12T00:18:13.860 DEBUG:teuthology.orchestra.run.smithi124:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-12T00:18:14.190 INFO:teuthology.provision.fog.smithi124:Node is ready 2024-01-12T00:18:14.190 DEBUG:teuthology.orchestra.run.smithi124:> hostname 2024-01-12T00:18:14.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:18:14.204 INFO:teuthology.orchestra.run.smithi124.stdout:smithi124.front.sepia.ceph.com 2024-01-12T00:18:14.205 DEBUG:teuthology.orchestra.run.smithi124:> grep smithi124.front.sepia.ceph.com /etc/hosts 2024-01-12T00:18:14.295 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.15.124 smithi124.front.sepia.ceph.com smithi124 2024-01-12T00:18:14.295 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124 2024-01-12T00:18:14.414 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-12T00:18:14.414 DEBUG:teuthology.orchestra.run.smithi188:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-12T00:18:14.439 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi124.front.sepia.ceph.com/smithi124/g' /etc/hosts 2024-01-12T00:18:14.466 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/172.21.15.124/172.21.15.124/g' /etc/hosts 2024-01-12T00:18:14.539 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i -e 's/smithi124.front.sepia.ceph.com/smithi124/g' /etc/hostname 2024-01-12T00:18:14.610 DEBUG:teuthology.orchestra.run.smithi124:> sudo hostname smithi124 2024-01-12T00:18:14.680 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2024-01-12T00:18:14.736 INFO:teuthology.provision.fog.smithi188:Node is ready 2024-01-12T00:18:14.737 DEBUG:teuthology.orchestra.run.smithi188:> hostname 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Stream" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="9" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="9" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el9" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:LOGO="fedora-logo-icon" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://centos.org/" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-12T00:18:14.738 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-12T00:18:14.739 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-12T00:18:14.739 INFO:teuthology.provision.fog.smithi124:Deploy complete! 2024-01-12T00:18:14.744 DEBUG:teuthology.parallel:result is None 2024-01-12T00:18:14.758 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188.front.sepia.ceph.com 2024-01-12T00:18:14.758 DEBUG:teuthology.orchestra.run.smithi188:> grep smithi188.front.sepia.ceph.com /etc/hosts 2024-01-12T00:18:14.811 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188 2024-01-12T00:18:14.811 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188 2024-01-12T00:18:14.882 INFO:teuthology.provision.fog.smithi107:Node is ready 2024-01-12T00:18:14.882 DEBUG:teuthology.orchestra.run.smithi107:> hostname 2024-01-12T00:18:14.900 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107.front.sepia.ceph.com 2024-01-12T00:18:14.900 DEBUG:teuthology.orchestra.run.smithi107:> grep smithi107.front.sepia.ceph.com /etc/hosts 2024-01-12T00:18:14.931 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188.front.sepia.ceph.com/smithi188/g' /etc/hosts 2024-01-12T00:18:14.957 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107 smithi107.front.sepia.ceph.com smithi107 2024-01-12T00:18:14.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname smithi107 2024-01-12T00:18:14.959 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/172.21.15.188/172.21.15.188/g' /etc/hosts 2024-01-12T00:18:15.026 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188.front.sepia.ceph.com/smithi188/g' /etc/hostname 2024-01-12T00:18:15.064 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/smithi107.front.sepia.ceph.com/smithi107/g' /etc/hosts 2024-01-12T00:18:15.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/172.21.15.107/172.21.15.107/g' /etc/hosts 2024-01-12T00:18:15.095 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188 2024-01-12T00:18:15.163 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-01-12T00:18:15.164 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/smithi107.front.sepia.ceph.com/smithi107/g' /etc/hostname 2024-01-12T00:18:15.218 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-01-12T00:18:15.218 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-01-12T00:18:15.218 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-01-12T00:18:15.218 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-01-12T00:18:15.218 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-12T00:18:15.219 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-12T00:18:15.219 INFO:teuthology.provision.fog.smithi188:Deploy complete! 2024-01-12T00:18:15.220 DEBUG:teuthology.parallel:result is None 2024-01-12T00:18:15.234 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname smithi107 2024-01-12T00:18:15.302 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-01-12T00:18:15.357 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-01-12T00:18:15.357 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-01-12T00:18:15.357 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-01-12T00:18:15.357 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-12T00:18:15.358 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-12T00:18:15.358 INFO:teuthology.provision.fog.smithi107:Deploy complete! 2024-01-12T00:18:15.359 DEBUG:teuthology.parallel:result is None 2024-01-12T00:18:15.599 INFO:teuthology.lock.ops:Checking smithi107.front.sepia.ceph.com 2024-01-12T00:18:15.600 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-12T00:18:15.624 INFO:teuthology.lock.ops:Checking smithi124.front.sepia.ceph.com 2024-01-12T00:18:15.624 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-12T00:18:15.649 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com 2024-01-12T00:18:15.649 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-12T00:18:15.667 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: set os type and version on server 2024-01-12T00:18:15.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-01-12T00:18:15.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:18:15.752 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-01-12T00:18:15.822 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-01-12T00:18:15.822 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-01-12T00:18:15.876 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-01-12T00:18:15.876 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-12T00:18:15.930 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-01-12T00:18:15.931 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-12T00:18:15.931 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-12T00:18:15.931 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-12T00:18:15.931 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-01-12T00:18:15.958 INFO:teuthology.lock.ops:Updating [smithi124.front.sepia.ceph.com]: set os type and version on server 2024-01-12T00:18:15.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-12T00:18:15.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:18:16.047 DEBUG:teuthology.orchestra.run.smithi124:> true 2024-01-12T00:18:16.113 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-12T00:18:16.114 DEBUG:teuthology.orchestra.run.smithi124:> uname -m 2024-01-12T00:18:16.169 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2024-01-12T00:18:16.170 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Stream" 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="9" 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="9" 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el9" 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2024-01-12T00:18:16.226 INFO:teuthology.orchestra.run.smithi124.stdout:LOGO="fedora-logo-icon" 2024-01-12T00:18:16.227 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-12T00:18:16.227 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://centos.org/" 2024-01-12T00:18:16.227 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-12T00:18:16.227 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-12T00:18:16.227 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-12T00:18:16.227 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2024-01-12T00:18:16.249 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server 2024-01-12T00:18:16.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-12T00:18:16.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-12T00:18:16.338 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-01-12T00:18:16.408 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-01-12T00:18:16.408 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-01-12T00:18:16.464 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-01-12T00:18:16.465 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-01-12T00:18:16.521 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-01-12T00:18:16.521 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-01-12T00:18:16.521 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-01-12T00:18:16.521 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-01-12T00:18:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-01-12T00:18:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-01-12T00:18:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-12T00:18:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-01-12T00:18:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-01-12T00:18:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-12T00:18:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-01-12T00:18:16.522 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-12T00:18:16.523 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-12T00:18:16.523 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-12T00:18:16.523 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-01-12T00:18:16.550 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:18:16.640 INFO:teuthology.dispatcher.supervisor:Running job 7513604 2024-01-12T00:18:16.640 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/bin/teuthology -v --owner scheduled_pdiazbou@teuthology --archive /home/teuthworker/archive/pdiazbou-2024-01-11_15:19:24-rados-buffer-space-onode-distro-default-smithi/7513604 --name pdiazbou-2024-01-11_15:19:24-rados-buffer-space-onode-distro-default-smithi --description rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_latest} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} -- /home/teuthworker/archive/pdiazbou-2024-01-11_15:19:24-rados-buffer-space-onode-distro-default-smithi/7513604/orig.config.yaml 2024-01-12T00:18:16.643 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdiazbou-2024-01-11_15:19:24-rados-buffer-space-onode-distro-default-smithi/7513604 2024-01-12T00:18:16.644 INFO:teuthology.dispatcher.supervisor:Job PID: 617208 2024-01-12T00:18:16.644 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-01-12T00:20:16.645 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:22:16.713 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:24:16.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:26:16.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:28:16.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:30:17.078 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:32:17.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:34:17.226 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:36:17.301 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:38:17.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:40:17.464 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-12T00:40:17.525 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-01-12T00:40:17.644 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2024-01-12T00:40:17.644 INFO:teuthology.nuke:Checking targets against current locks 2024-01-12T00:40:17.697 DEBUG:teuthology.nuke:shortname: smithi107 2024-01-12T00:40:17.697 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-12T00:40:17.718 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdiazbou-2024-01-11_15:19:24-rados-buffer-space-onode-distro-default-smithi/7513604', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-12 00:13:17.019547', 'locked_by': 'scheduled_pdiazbou@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMVEfhgpXD0j7sd1UQ2J+L37GHYU/9rTWoNBI0jAkty6yxO4gp6/4mObNfkHI2gJtlyR5QHGIxmgzIzi8i4GhuE='} 2024-01-12T00:40:17.722 DEBUG:teuthology.nuke:shortname: smithi124 2024-01-12T00:40:17.722 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-12T00:40:17.740 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdiazbou-2024-01-11_15:19:24-rados-buffer-space-onode-distro-default-smithi/7513604', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-12 00:13:17.018274', 'locked_by': 'scheduled_pdiazbou@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAOGsXzlNLbW/KjTK0Z0B25ZfyYruLLNKCkkfQoWrkp4J20ed6gBLTTm3DGm+G+3/bM94Nr2jdGqautst6xezmE='} 2024-01-12T00:40:17.743 DEBUG:teuthology.nuke:shortname: smithi188 2024-01-12T00:40:17.744 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-12T00:40:17.765 INFO:teuthology.orchestra.console.smithi107:Power off 2024-01-12T00:40:17.766 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-12T00:40:17.777 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdiazbou-2024-01-11_15:19:24-rados-buffer-space-onode-distro-default-smithi/7513604', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-12 00:13:17.016789', 'locked_by': 'scheduled_pdiazbou@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0/wyuHoUs3bK3+ZksaYb/y/YxQtj1fPBC2PA+Wycjy9rsGfawE9QP6S2/S0RJG3zdwxkvi0k/qKancdJNzr5A='} 2024-01-12T00:40:17.788 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-01-12T00:40:17.788 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:40:17.811 INFO:teuthology.orchestra.console.smithi124:Power off 2024-01-12T00:40:17.811 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-12T00:40:17.820 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-01-12T00:40:17.833 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2024-01-12T00:40:17.833 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:40:17.850 INFO:teuthology.orchestra.console.smithi188:Power off 2024-01-12T00:40:17.850 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-12T00:40:17.859 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2024-01-12T00:40:17.872 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-01-12T00:40:17.872 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-12T00:40:17.894 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-01-12T00:40:17.921 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-01-12T00:40:17.960 INFO:teuthology.orchestra.console.smithi124:Power off completed 2024-01-12T00:40:17.995 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-01-12T00:40:18.076 INFO:teuthology.lock.ops:unlocked: smithi107.front.sepia.ceph.com 2024-01-12T00:40:18.110 INFO:teuthology.lock.ops:unlocked: smithi124.front.sepia.ceph.com 2024-01-12T00:40:18.141 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com