2024-01-22T04:46:29.386 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-22T04:46:29.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-22T04:46:29.522 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fed4a73ac10>, signals=[15]) 2024-01-22T04:46:29.522 INFO:teuthology.lock.ops:Start node 'smithi017.front.sepia.ceph.com' reimaging 2024-01-22T04:46:29.522 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: reset os type and version on server 2024-01-22T04:46:29.522 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-01-22T04:46:29.539 INFO:teuthology.lock.ops:Start node 'smithi153.front.sepia.ceph.com' reimaging 2024-01-22T04:46:29.539 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: reset os type and version on server 2024-01-22T04:46:29.539 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-01-22T04:46:29.656 DEBUG:teuthology.provision.fog.smithi017:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-22T04:46:29.672 DEBUG:teuthology.provision.fog.smithi153:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-22T04:46:29.701 INFO:teuthology.provision.fog.smithi017:Scheduling deploy of rhel 8.6 2024-01-22T04:46:29.711 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of rhel 8.6 2024-01-22T04:46:29.931 INFO:teuthology.orchestra.console.smithi017:Power off 2024-01-22T04:46:29.932 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-22T04:46:29.944 INFO:teuthology.orchestra.console.smithi153:Power off 2024-01-22T04:46:29.944 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-22T04:46:29.956 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off 2024-01-22T04:46:29.957 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:29.966 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off 2024-01-22T04:46:29.967 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:29.978 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-01-22T04:46:29.988 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-01-22T04:46:33.984 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:33.990 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:34.092 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-01-22T04:46:34.097 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-01-22T04:46:38.096 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:38.102 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:38.204 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-01-22T04:46:38.213 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-01-22T04:46:42.210 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:42.217 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:42.319 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-01-22T04:46:42.325 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2024-01-22T04:46:42.420 INFO:teuthology.orchestra.console.smithi017:Power off completed 2024-01-22T04:46:42.426 INFO:teuthology.orchestra.console.smithi153:Power off completed 2024-01-22T04:46:42.522 INFO:teuthology.orchestra.console.smithi017:Power on 2024-01-22T04:46:42.522 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-22T04:46:42.530 INFO:teuthology.orchestra.console.smithi153:Power on 2024-01-22T04:46:42.530 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-22T04:46:42.543 DEBUG:teuthology.orchestra.console.smithi017:power on output: Chassis Power Control: Up/On 2024-01-22T04:46:42.543 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:42.551 DEBUG:teuthology.orchestra.console.smithi153:power on output: Chassis Power Control: Up/On 2024-01-22T04:46:42.552 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:42.565 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-01-22T04:46:42.572 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2024-01-22T04:46:46.571 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:46.578 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:46.680 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-01-22T04:46:46.686 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2024-01-22T04:46:50.685 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:50.692 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T04:46:50.794 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-01-22T04:46:50.800 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is on 2024-01-22T04:46:50.896 INFO:teuthology.orchestra.console.smithi017:Power on completed 2024-01-22T04:46:50.902 INFO:teuthology.orchestra.console.smithi153:Power on completed 2024-01-22T04:46:51.000 INFO:teuthology.provision.fog.smithi017:Waiting for deploy to finish 2024-01-22T04:46:51.004 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2024-01-22T04:46:51.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:47:09.628 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:47:15.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:47:18.684 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:47:24.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:47:27.744 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:47:33.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:47:36.828 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:47:42.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:47:45.884 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:47:51.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:47:54.940 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:48:00.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:48:03.996 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:48:10.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:48:20.284 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:48:26.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:48:29.340 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:48:35.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:48:35.475 ERROR:teuthology.orchestra.connection:Error authenticating with smithi153.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-22T04:48:39.602 ERROR:teuthology.orchestra.connection:Error authenticating with smithi153.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-22T04:48:46.688 ERROR:teuthology.orchestra.connection:Error authenticating with smithi153.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-22T04:48:56.776 ERROR:teuthology.orchestra.connection:Error authenticating with smithi153.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-22T04:49:09.900 ERROR:teuthology.orchestra.connection:Error authenticating with smithi153.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-22T04:49:21.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:49:26.025 ERROR:teuthology.orchestra.connection:Error authenticating with smithi153.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-01-22T04:49:40.156 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-01-22T04:49:46.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:49:49.212 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-01-22T04:49:55.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:49:58.272 WARNING:teuthology.provision.fog.smithi017:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-01-22T04:50:04.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:50:45.028 WARNING:teuthology.provision.fog.smithi153:timed out 2024-01-22T04:50:51.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:50:54.108 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:51:00.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:51:03.164 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:51:04.276 WARNING:teuthology.provision.fog.smithi017:timed out 2024-01-22T04:51:09.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:51:09.308 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:51:10.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:51:10.493 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-22T04:51:10.494 DEBUG:teuthology.orchestra.run.smithi017:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-22T04:51:10.759 INFO:teuthology.provision.fog.smithi017:Node is ready 2024-01-22T04:51:10.760 DEBUG:teuthology.orchestra.run.smithi017:> hostname 2024-01-22T04:51:10.780 INFO:teuthology.orchestra.run.smithi017.stdout:smithi017.front.sepia.ceph.com 2024-01-22T04:51:10.780 DEBUG:teuthology.orchestra.run.smithi017:> grep smithi017.front.sepia.ceph.com /etc/hosts 2024-01-22T04:51:10.841 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.15.17 smithi017.front.sepia.ceph.com smithi017 2024-01-22T04:51:10.841 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname smithi017 2024-01-22T04:51:10.957 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/smithi017.front.sepia.ceph.com/smithi017/g' /etc/hosts 2024-01-22T04:51:10.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/172.21.15.17/172.21.15.17/g' /etc/hosts 2024-01-22T04:51:11.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i -e 's/smithi017.front.sepia.ceph.com/smithi017/g' /etc/hostname 2024-01-22T04:51:11.140 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname smithi017 2024-01-22T04:51:11.216 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-01-22T04:51:11.278 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Red Hat Enterprise Linux" 2024-01-22T04:51:11.278 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8.6 (Ootpa)" 2024-01-22T04:51:11.278 INFO:teuthology.orchestra.run.smithi017.stdout:ID="rhel" 2024-01-22T04:51:11.278 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="fedora" 2024-01-22T04:51:11.278 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8.6" 2024-01-22T04:51:11.278 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.redhat.com/" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-22T04:51:11.279 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-22T04:51:11.280 INFO:teuthology.provision.fog.smithi017:Deploy complete! 2024-01-22T04:51:11.281 DEBUG:teuthology.parallel:result is None 2024-01-22T04:51:15.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:51:18.368 WARNING:teuthology.provision.fog.smithi153:[Errno None] Unable to connect to port 22 on 172.21.15.153 2024-01-22T04:51:24.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:51:24.593 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-22T04:51:24.593 DEBUG:teuthology.orchestra.run.smithi153:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-22T04:51:44.847 INFO:teuthology.provision.fog.smithi153:Node is ready 2024-01-22T04:51:44.848 DEBUG:teuthology.orchestra.run.smithi153:> hostname 2024-01-22T04:51:44.870 INFO:teuthology.orchestra.run.smithi153.stdout:smithi153.front.sepia.ceph.com 2024-01-22T04:51:44.870 DEBUG:teuthology.orchestra.run.smithi153:> grep smithi153.front.sepia.ceph.com /etc/hosts 2024-01-22T04:51:44.930 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.153 smithi153.front.sepia.ceph.com smithi153 2024-01-22T04:51:44.930 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname smithi153 2024-01-22T04:51:45.003 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi153.front.sepia.ceph.com/smithi153/g' /etc/hosts 2024-01-22T04:51:45.073 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/172.21.15.153/172.21.15.153/g' /etc/hosts 2024-01-22T04:51:45.143 DEBUG:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi153.front.sepia.ceph.com/smithi153/g' /etc/hostname 2024-01-22T04:51:45.212 DEBUG:teuthology.orchestra.run.smithi153:> sudo hostname smithi153 2024-01-22T04:51:45.280 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-01-22T04:51:45.337 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Red Hat Enterprise Linux" 2024-01-22T04:51:45.337 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8.6 (Ootpa)" 2024-01-22T04:51:45.337 INFO:teuthology.orchestra.run.smithi153.stdout:ID="rhel" 2024-01-22T04:51:45.337 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="fedora" 2024-01-22T04:51:45.337 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8.6" 2024-01-22T04:51:45.337 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2024-01-22T04:51:45.337 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-22T04:51:45.337 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.redhat.com/" 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-22T04:51:45.338 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-22T04:51:45.338 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2024-01-22T04:51:45.339 DEBUG:teuthology.parallel:result is None 2024-01-22T04:51:45.493 INFO:teuthology.lock.ops:Checking smithi017.front.sepia.ceph.com 2024-01-22T04:51:45.494 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-22T04:51:45.519 INFO:teuthology.lock.ops:Checking smithi153.front.sepia.ceph.com 2024-01-22T04:51:45.520 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-22T04:51:45.545 INFO:teuthology.lock.ops:Updating [smithi017.front.sepia.ceph.com]: set os type and version on server 2024-01-22T04:51:45.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-01-22T04:51:45.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:51:45.619 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-01-22T04:51:45.765 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-01-22T04:51:45.765 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2024-01-22T04:51:45.826 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2024-01-22T04:51:45.826 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-01-22T04:51:45.887 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Red Hat Enterprise Linux" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8.6 (Ootpa)" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:ID="rhel" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="fedora" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8.6" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.redhat.com/" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-22T04:51:45.888 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-01-22T04:51:45.889 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-22T04:51:45.889 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-22T04:51:45.889 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-22T04:51:45.889 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-22T04:51:45.889 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-01-22T04:51:45.911 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2024-01-22T04:51:45.912 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-01-22T04:51:45.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi153.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-22T04:51:45.983 DEBUG:teuthology.orchestra.run.smithi153:> true 2024-01-22T04:51:46.049 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi153.front.sepia.ceph.com' 2024-01-22T04:51:46.049 DEBUG:teuthology.orchestra.run.smithi153:> uname -m 2024-01-22T04:51:46.106 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2024-01-22T04:51:46.106 DEBUG:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="Red Hat Enterprise Linux" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="8.6 (Ootpa)" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:ID="rhel" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="fedora" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="8.6" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:PLATFORM_ID="platform:el8" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-22T04:51:46.163 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.redhat.com/" 2024-01-22T04:51:46.164 INFO:teuthology.orchestra.run.smithi153.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-22T04:51:46.164 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-22T04:51:46.164 INFO:teuthology.orchestra.run.smithi153.stdout: 2024-01-22T04:51:46.164 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-22T04:51:46.164 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-22T04:51:46.164 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-22T04:51:46.164 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-22T04:51:46.165 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2024-01-22T04:51:46.186 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-22T04:51:46.277 INFO:teuthology.dispatcher.supervisor:Running job 7523569 2024-01-22T04:51:46.277 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-01-19_15:47:44-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7523569 --name vshankar-2024-01-19_15:47:44-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} -- /home/teuthworker/archive/vshankar-2024-01-19_15:47:44-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7523569/orig.config.yaml 2024-01-22T04:51:46.280 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-01-19_15:47:44-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7523569 2024-01-22T04:51:46.280 INFO:teuthology.dispatcher.supervisor:Job PID: 3399543 2024-01-22T04:51:46.280 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-01-22T04:53:46.282 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-22T04:55:46.365 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-22T04:57:46.430 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-22T04:59:46.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-22T05:01:46.573 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-22T05:01:46.655 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-01-22T05:01:46.760 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi153 but it was locked by another job: /home/teuthworker/archive/vshankar-2024-01-19_15:47:44-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7523570 2024-01-22T05:01:46.760 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2024-01-22T05:01:46.760 INFO:teuthology.nuke:Checking targets against current locks 2024-01-22T05:01:46.807 DEBUG:teuthology.nuke:shortname: smithi017 2024-01-22T05:01:46.808 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-22T05:01:46.828 DEBUG:teuthology.nuke:shortname: smithi153 2024-01-22T05:01:46.828 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-22T05:01:46.831 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_15:47:44-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7523569', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-22 04:46:28.359857', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN/k5KMg9RUWsrr5CV0kmc04mHt7p+g6CoUSrmLyc1qM0bm/we/MVQdGqQ1Kh++wxE4G4GePP2rAmLe6nEyH1hc='} 2024-01-22T05:01:46.853 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi153.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_15:47:44-fs-wip-vshankar-testing-pacific-020240119.072505-testing-default-smithi/7523570', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-22 04:46:28.754628', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFaAoL4BtIglF9B5qEnDwSoTVHT18mv7qy5s1q0vcgcCNXY0mCbfwn2xuc9KBW+TBFH4p+HaFXgDItmEPKLyZnM='} 2024-01-22T05:01:46.871 INFO:teuthology.orchestra.console.smithi017:Power off 2024-01-22T05:01:46.871 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-22T05:01:46.893 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off 2024-01-22T05:01:46.894 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T05:01:46.907 INFO:teuthology.orchestra.console.smithi153:Power off 2024-01-22T05:01:46.907 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-22T05:01:46.916 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-01-22T05:01:46.928 DEBUG:teuthology.orchestra.console.smithi153:power off output: Chassis Power Control: Down/Off 2024-01-22T05:01:46.928 DEBUG:teuthology.orchestra.console.smithi153:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-22T05:01:46.949 DEBUG:teuthology.orchestra.console.smithi153:check power output: Chassis Power is off 2024-01-22T05:01:47.017 INFO:teuthology.orchestra.console.smithi017:Power off completed 2024-01-22T05:01:47.049 INFO:teuthology.orchestra.console.smithi153:Power off completed 2024-01-22T05:01:47.171 INFO:teuthology.lock.ops:unlocked: smithi017.front.sepia.ceph.com 2024-01-22T05:01:47.201 INFO:teuthology.lock.ops:unlocked: smithi153.front.sepia.ceph.com