2023-12-19T18:00:07.756 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-19T18:00:07.757 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:00:07.929 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f39f5d50940>, signals=[15]) 2023-12-19T18:00:07.929 INFO:teuthology.lock.ops:Start node 'smithi087.front.sepia.ceph.com' reimaging 2023-12-19T18:00:07.930 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: reset os type and version on server 2023-12-19T18:00:07.930 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-12-19T18:00:07.965 INFO:teuthology.lock.ops:Start node 'smithi162.front.sepia.ceph.com' reimaging 2023-12-19T18:00:07.965 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: reset os type and version on server 2023-12-19T18:00:07.966 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-12-19T18:00:08.125 DEBUG:teuthology.provision.fog.smithi087:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-19T18:00:08.154 DEBUG:teuthology.provision.fog.smithi162:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-19T18:00:08.177 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of rhel 8.6 2023-12-19T18:00:08.209 INFO:teuthology.provision.fog.smithi162:Scheduling deploy of rhel 8.6 2023-12-19T18:00:08.565 INFO:teuthology.orchestra.console.smithi162:Power off 2023-12-19T18:00:08.566 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T18:00:08.579 INFO:teuthology.orchestra.console.smithi087:Power off 2023-12-19T18:00:08.579 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T18:00:08.614 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2023-12-19T18:00:08.615 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:08.623 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2023-12-19T18:00:08.624 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:08.663 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-12-19T18:00:08.671 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-12-19T18:00:08.771 INFO:teuthology.orchestra.console.smithi087:Power off completed 2023-12-19T18:00:08.872 INFO:teuthology.orchestra.console.smithi087:Power on 2023-12-19T18:00:08.872 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-19T18:00:08.901 DEBUG:teuthology.orchestra.console.smithi087:power on output: Chassis Power Control: Up/On 2023-12-19T18:00:08.901 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:08.948 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-12-19T18:00:12.668 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:12.777 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-12-19T18:00:12.949 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:13.060 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-12-19T18:00:16.781 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:16.889 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-12-19T18:00:16.990 INFO:teuthology.orchestra.console.smithi162:Power off completed 2023-12-19T18:00:17.060 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:17.091 INFO:teuthology.orchestra.console.smithi162:Power on 2023-12-19T18:00:17.092 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-19T18:00:17.121 DEBUG:teuthology.orchestra.console.smithi162:power on output: Chassis Power Control: Up/On 2023-12-19T18:00:17.122 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:17.167 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-12-19T18:00:17.169 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-12-19T18:00:17.269 INFO:teuthology.orchestra.console.smithi087:Power on completed 2023-12-19T18:00:17.370 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish 2023-12-19T18:00:21.169 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:21.279 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-12-19T18:00:25.281 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:00:25.420 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is on 2023-12-19T18:00:25.521 INFO:teuthology.orchestra.console.smithi162:Power on completed 2023-12-19T18:00:25.622 INFO:teuthology.provision.fog.smithi162:Waiting for deploy to finish 2023-12-19T18:03:03.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:06.742 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-19T18:03:12.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:15.829 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-19T18:03:21.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:24.913 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-19T18:03:27.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:30.193 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-12-19T18:03:30.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:34.006 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-19T18:03:36.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:39.285 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-12-19T18:03:40.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:43.089 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-19T18:03:45.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:48.369 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-12-19T18:03:49.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:54.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:03:57.461 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-12-19T18:04:03.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:04:06.549 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-12-19T18:04:12.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:04:15.638 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-12-19T18:04:21.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:04:24.721 WARNING:teuthology.provision.fog.smithi162:[Errno None] Unable to connect to port 22 on 172.21.15.162 2023-12-19T18:04:30.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:04:49.118 WARNING:teuthology.provision.fog.smithi087:timed out 2023-12-19T18:04:55.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:04:55.385 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-19T18:04:55.385 DEBUG:teuthology.orchestra.run.smithi087:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-19T18:04:55.648 INFO:teuthology.provision.fog.smithi087:Node is ready 2023-12-19T18:04:55.648 DEBUG:teuthology.orchestra.run.smithi087:> hostname 2023-12-19T18:04:55.706 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087 2023-12-19T18:04:55.707 DEBUG:teuthology.orchestra.run.smithi087:> grep smithi087 /etc/hosts 2023-12-19T18:04:55.765 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2023-12-19T18:04:55.765 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2023-12-19T18:04:55.868 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087/smithi087/g' /etc/hosts 2023-12-19T18:04:55.897 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/172.21.15.87/172.21.15.87/g' /etc/hosts 2023-12-19T18:04:55.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087/smithi087/g' /etc/hostname 2023-12-19T18:04:56.035 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2023-12-19T18:04:56.102 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-12-19T18:04:56.159 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T18:04:56.159 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.6 (Ootpa)" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.6" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-19T18:04:56.160 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T18:04:56.161 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-19T18:04:56.162 INFO:teuthology.provision.fog.smithi087:Deploy complete! 2023-12-19T18:04:56.163 DEBUG:teuthology.parallel:result is None 2023-12-19T18:05:30.770 WARNING:teuthology.provision.fog.smithi162:timed out 2023-12-19T18:05:36.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:05:36.969 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-19T18:05:36.969 DEBUG:teuthology.orchestra.run.smithi162:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-19T18:05:37.244 INFO:teuthology.provision.fog.smithi162:Node is ready 2023-12-19T18:05:37.244 DEBUG:teuthology.orchestra.run.smithi162:> hostname 2023-12-19T18:05:37.262 INFO:teuthology.orchestra.run.smithi162.stdout:smithi162 2023-12-19T18:05:37.263 DEBUG:teuthology.orchestra.run.smithi162:> grep smithi162 /etc/hosts 2023-12-19T18:05:37.318 INFO:teuthology.orchestra.run.smithi162.stdout:172.21.15.162 smithi162.front.sepia.ceph.com smithi162 2023-12-19T18:05:37.319 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname smithi162 2023-12-19T18:05:37.442 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/smithi162/smithi162/g' /etc/hosts 2023-12-19T18:05:37.471 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/172.21.15.162/172.21.15.162/g' /etc/hosts 2023-12-19T18:05:37.537 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i -e 's/smithi162/smithi162/g' /etc/hostname 2023-12-19T18:05:37.602 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname smithi162 2023-12-19T18:05:37.668 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="8.6 (Ootpa)" 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:ID="rhel" 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="fedora" 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="8.6" 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el8" 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2023-12-19T18:05:37.722 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-19T18:05:37.723 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T18:05:37.723 INFO:teuthology.orchestra.run.smithi162.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T18:05:37.723 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T18:05:37.723 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-19T18:05:37.723 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T18:05:37.723 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-19T18:05:37.723 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T18:05:37.723 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-19T18:05:37.723 INFO:teuthology.provision.fog.smithi162:Deploy complete! 2023-12-19T18:05:37.724 DEBUG:teuthology.parallel:result is None 2023-12-19T18:05:38.044 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com 2023-12-19T18:05:38.045 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-19T18:05:38.079 INFO:teuthology.lock.ops:Checking smithi162.front.sepia.ceph.com 2023-12-19T18:05:38.079 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-19T18:05:38.113 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server 2023-12-19T18:05:38.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-12-19T18:05:38.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:05:38.208 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-12-19T18:05:38.344 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-12-19T18:05:38.344 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2023-12-19T18:05:38.402 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2023-12-19T18:05:38.402 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-12-19T18:05:38.460 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T18:05:38.460 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.6 (Ootpa)" 2023-12-19T18:05:38.461 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2023-12-19T18:05:38.461 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2023-12-19T18:05:38.461 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.6" 2023-12-19T18:05:38.461 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-12-19T18:05:38.461 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-19T18:05:38.461 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-12-19T18:05:38.461 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-19T18:05:38.462 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T18:05:38.462 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T18:05:38.462 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T18:05:38.462 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-12-19T18:05:38.462 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T18:05:38.463 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-19T18:05:38.463 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T18:05:38.463 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-19T18:05:38.464 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-12-19T18:05:38.496 INFO:teuthology.lock.ops:Updating [smithi162.front.sepia.ceph.com]: set os type and version on server 2023-12-19T18:05:38.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-12-19T18:05:38.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:05:38.573 DEBUG:teuthology.orchestra.run.smithi162:> true 2023-12-19T18:05:38.648 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2023-12-19T18:05:38.649 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2023-12-19T18:05:38.708 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2023-12-19T18:05:38.708 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2023-12-19T18:05:38.766 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T18:05:38.766 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="8.6 (Ootpa)" 2023-12-19T18:05:38.766 INFO:teuthology.orchestra.run.smithi162.stdout:ID="rhel" 2023-12-19T18:05:38.767 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="fedora" 2023-12-19T18:05:38.767 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="8.6" 2023-12-19T18:05:38.767 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el8" 2023-12-19T18:05:38.767 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-19T18:05:38.767 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2023-12-19T18:05:38.767 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-19T18:05:38.767 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T18:05:38.767 INFO:teuthology.orchestra.run.smithi162.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T18:05:38.768 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T18:05:38.768 INFO:teuthology.orchestra.run.smithi162.stdout: 2023-12-19T18:05:38.768 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T18:05:38.768 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-19T18:05:38.768 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T18:05:38.768 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-19T18:05:38.769 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2023-12-19T18:05:38.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:05:38.902 INFO:teuthology.dispatcher.supervisor:Running job 7497240 2023-12-19T18:05:38.903 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497240 --name vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} -- /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497240/orig.config.yaml 2023-12-19T18:05:38.905 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497240 2023-12-19T18:05:38.906 INFO:teuthology.dispatcher.supervisor:Job PID: 1097710 2023-12-19T18:05:38.906 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-19T18:07:39.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:09:39.190 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:11:39.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:13:39.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:15:39.698 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:17:39.870 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:19:40.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:21:40.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:21:40.303 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-19T18:21:40.987 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-19T18:21:40.987 INFO:teuthology.nuke:Checking targets against current locks 2023-12-19T18:21:41.079 DEBUG:teuthology.nuke:shortname: smithi087 2023-12-19T18:21:41.079 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T18:21:41.115 DEBUG:teuthology.nuke:shortname: smithi162 2023-12-19T18:21:41.115 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T18:21:41.123 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497240', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 18:00:06.918448', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWrRSb4A11ORC2DtjikRufgaHqi0h8tixiRKarvnr7TThZnAKeDmAFGkeA+ihnsD1EFYgs0sbAtiJLVlCeLaeQ='} 2023-12-19T18:21:41.168 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497240', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 18:00:06.916738', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKXpxTq0XEeK0e1Hlo2p9spBsEhKn3Sbqw0jWg89Gn/GK6L88gge4IbzyFTS58SNHYx5MEJ+rTLenOdgFXs6QLs='} 2023-12-19T18:21:41.220 INFO:teuthology.orchestra.console.smithi087:Power off 2023-12-19T18:21:41.220 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T18:21:41.260 INFO:teuthology.orchestra.console.smithi162:Power off 2023-12-19T18:21:41.261 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T18:21:41.269 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2023-12-19T18:21:41.270 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:21:41.299 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-12-19T18:21:41.308 DEBUG:teuthology.orchestra.console.smithi162:power off output: Chassis Power Control: Down/Off 2023-12-19T18:21:41.308 DEBUG:teuthology.orchestra.console.smithi162:pexpect command: ipmitool -H smithi162.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:21:41.354 DEBUG:teuthology.orchestra.console.smithi162:check power output: Chassis Power is off 2023-12-19T18:21:41.400 INFO:teuthology.orchestra.console.smithi087:Power off completed 2023-12-19T18:21:41.456 INFO:teuthology.orchestra.console.smithi162:Power off completed 2023-12-19T18:21:41.568 INFO:teuthology.lock.ops:unlocked: smithi087.front.sepia.ceph.com 2023-12-19T18:21:41.624 INFO:teuthology.lock.ops:unlocked: smithi162.front.sepia.ceph.com