2023-12-22T09:25:04.800 INFO:root:teuthology version: 0.0.1.dev226+g9680d2a 2023-12-22T09:25:04.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:25:04.958 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f26eee8b940>, signals=[15]) 2023-12-22T09:25:04.958 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging 2023-12-22T09:25:04.959 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server 2023-12-22T09:25:04.959 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-12-22T09:25:04.978 INFO:teuthology.lock.ops:Start node 'smithi123.front.sepia.ceph.com' reimaging 2023-12-22T09:25:04.979 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: reset os type and version on server 2023-12-22T09:25:04.979 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-12-22T09:25:05.101 DEBUG:teuthology.provision.fog.smithi016:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-22T09:25:05.106 DEBUG:teuthology.provision.fog.smithi123:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-22T09:25:05.146 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of rhel 8.4 2023-12-22T09:25:05.149 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of rhel 8.4 2023-12-22T09:25:05.447 INFO:teuthology.orchestra.console.smithi123:Power off 2023-12-22T09:25:05.448 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T09:25:05.465 INFO:teuthology.orchestra.console.smithi016:Power off 2023-12-22T09:25:05.465 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T09:25:05.474 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-12-22T09:25:05.475 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:05.486 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-12-22T09:25:05.487 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:05.496 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-12-22T09:25:05.507 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-12-22T09:25:05.596 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-12-22T09:25:05.608 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-12-22T09:25:05.698 INFO:teuthology.orchestra.console.smithi123:Power on 2023-12-22T09:25:05.698 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-22T09:25:05.709 INFO:teuthology.orchestra.console.smithi016:Power on 2023-12-22T09:25:05.710 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-22T09:25:05.720 DEBUG:teuthology.orchestra.console.smithi123:power on output: Chassis Power Control: Up/On 2023-12-22T09:25:05.720 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:05.732 DEBUG:teuthology.orchestra.console.smithi016:power on output: Chassis Power Control: Up/On 2023-12-22T09:25:05.732 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:05.741 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-12-22T09:25:05.755 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-12-22T09:25:09.745 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:09.755 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:09.856 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-12-22T09:25:09.864 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-12-22T09:25:09.965 INFO:teuthology.orchestra.console.smithi016:Power on completed 2023-12-22T09:25:10.065 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2023-12-22T09:25:13.860 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:25:13.970 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-12-22T09:25:14.071 INFO:teuthology.orchestra.console.smithi123:Power on completed 2023-12-22T09:25:14.171 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish 2023-12-22T09:27:56.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:27:59.089 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-22T09:28:00.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:03.157 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-12-22T09:28:05.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:08.145 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-22T09:28:09.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:12.209 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-12-22T09:28:14.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:17.201 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-22T09:28:18.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:21.269 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-12-22T09:28:23.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:26.257 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-22T09:28:27.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:30.353 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-12-22T09:28:32.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:35.313 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-22T09:28:36.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:39.409 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-12-22T09:28:41.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:45.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:28:48.467 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-12-22T09:28:54.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:29:41.366 WARNING:teuthology.provision.fog.smithi016:timed out 2023-12-22T09:29:47.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:29:47.740 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-22T09:29:47.741 DEBUG:teuthology.orchestra.run.smithi016:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-22T09:29:48.002 INFO:teuthology.provision.fog.smithi016:Node is ready 2023-12-22T09:29:48.002 DEBUG:teuthology.orchestra.run.smithi016:> hostname 2023-12-22T09:29:48.019 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com 2023-12-22T09:29:48.019 DEBUG:teuthology.orchestra.run.smithi016:> grep smithi016.front.sepia.ceph.com /etc/hosts 2023-12-22T09:29:48.077 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016 2023-12-22T09:29:48.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2023-12-22T09:29:48.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hosts 2023-12-22T09:29:48.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/172.21.15.16/172.21.15.16/g' /etc/hosts 2023-12-22T09:29:48.276 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hostname 2023-12-22T09:29:48.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2023-12-22T09:29:48.407 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-12-22T09:29:48.464 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T09:29:48.464 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.4 (Ootpa)" 2023-12-22T09:29:48.464 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.4" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T09:29:48.465 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-22T09:29:48.469 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2023-12-22T09:29:48.471 DEBUG:teuthology.parallel:result is None 2023-12-22T09:29:54.481 WARNING:teuthology.provision.fog.smithi123:timed out 2023-12-22T09:30:00.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:30:00.731 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-22T09:30:00.731 DEBUG:teuthology.orchestra.run.smithi123:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-22T09:30:01.033 INFO:teuthology.provision.fog.smithi123:Node is ready 2023-12-22T09:30:01.033 DEBUG:teuthology.orchestra.run.smithi123:> hostname 2023-12-22T09:30:01.089 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com 2023-12-22T09:30:01.090 DEBUG:teuthology.orchestra.run.smithi123:> grep smithi123.front.sepia.ceph.com /etc/hosts 2023-12-22T09:30:01.145 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123 2023-12-22T09:30:01.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname smithi123 2023-12-22T09:30:01.260 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/smithi123.front.sepia.ceph.com/smithi123/g' /etc/hosts 2023-12-22T09:30:01.288 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/172.21.15.123/172.21.15.123/g' /etc/hosts 2023-12-22T09:30:01.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/smithi123.front.sepia.ceph.com/smithi123/g' /etc/hostname 2023-12-22T09:30:01.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname smithi123 2023-12-22T09:30:01.482 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-12-22T09:30:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T09:30:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.4 (Ootpa)" 2023-12-22T09:30:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2023-12-22T09:30:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2023-12-22T09:30:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.4" 2023-12-22T09:30:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2023-12-22T09:30:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-22T09:30:01.538 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T09:30:01.539 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-22T09:30:01.539 INFO:teuthology.provision.fog.smithi123:Deploy complete! 2023-12-22T09:30:01.540 DEBUG:teuthology.parallel:result is None 2023-12-22T09:30:01.691 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2023-12-22T09:30:01.692 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-22T09:30:01.738 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com 2023-12-22T09:30:01.738 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-22T09:30:01.764 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2023-12-22T09:30:01.764 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-22T09:30:01.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:30:01.835 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-12-22T09:30:01.953 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-22T09:30:01.953 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-12-22T09:30:02.008 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-12-22T09:30:02.009 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.4 (Ootpa)" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.4" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-22T09:30:02.065 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T09:30:02.066 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T09:30:02.066 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T09:30:02.066 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-22T09:30:02.066 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T09:30:02.066 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-22T09:30:02.066 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T09:30:02.066 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-22T09:30:02.066 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-12-22T09:30:02.096 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server 2023-12-22T09:30:02.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-12-22T09:30:02.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T09:30:02.162 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-12-22T09:30:02.228 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-12-22T09:30:02.228 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-12-22T09:30:02.282 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-12-22T09:30:02.282 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-12-22T09:30:02.337 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T09:30:02.337 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8.4 (Ootpa)" 2023-12-22T09:30:02.337 INFO:teuthology.orchestra.run.smithi123.stdout:ID="rhel" 2023-12-22T09:30:02.337 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="fedora" 2023-12-22T09:30:02.337 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8.4" 2023-12-22T09:30:02.337 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T09:30:02.338 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-22T09:30:02.339 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-12-22T09:30:02.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:30:02.470 INFO:teuthology.dispatcher.supervisor:Running job 7499456 2023-12-22T09:30:02.471 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9680d2aba1e76f64ade295c81599c97907457848/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499456 --name rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} -- /home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499456/orig.config.yaml 2023-12-22T09:30:02.476 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499456 2023-12-22T09:30:02.476 INFO:teuthology.dispatcher.supervisor:Job PID: 2256549 2023-12-22T09:30:02.476 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-22T09:32:02.574 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:34:02.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:36:02.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:38:03.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:40:03.266 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T09:40:03.340 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-22T09:40:03.465 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-22T09:40:03.465 INFO:teuthology.nuke:Checking targets against current locks 2023-12-22T09:40:03.515 DEBUG:teuthology.nuke:shortname: smithi016 2023-12-22T09:40:03.515 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-22T09:40:03.535 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-22 09:25:03.938164', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBObsZ8rtIGIEwz+GllH4w30p08gNMtajAIUvtqdMQNJbq9cFXHjhfOpjUhTI4JUW6XIPWNU6RK8RqKyh/cRidXc='} 2023-12-22T09:40:03.538 DEBUG:teuthology.nuke:shortname: smithi123 2023-12-22T09:40:03.539 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-22T09:40:03.558 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-22_08:14:05-fs-rishabh-mdscap-quincy-testing-default-smithi/7499456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-22 09:25:03.940146', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCcm/1BF6W07b3xjISHwABC92KQbg4bw61JbBU8ozFOB8TZP0mUjRTXfrMCMa3lgEQGLb2Shx/GOb1smpWDFfIA='} 2023-12-22T09:40:03.589 INFO:teuthology.orchestra.console.smithi016:Power off 2023-12-22T09:40:03.589 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T09:40:03.611 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-12-22T09:40:03.612 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:40:03.628 INFO:teuthology.orchestra.console.smithi123:Power off 2023-12-22T09:40:03.628 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T09:40:03.636 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-12-22T09:40:03.650 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-12-22T09:40:03.650 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T09:40:03.673 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-12-22T09:40:03.738 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-12-22T09:40:03.926 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-12-22T09:40:04.071 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com 2023-12-22T09:40:04.079 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com