2023-12-19T15:18:10.797 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-19T15:18:10.797 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T15:18:10.936 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc0a3034940>, signals=[15]) 2023-12-19T15:18:10.937 INFO:teuthology.lock.ops:Start node 'smithi012.front.sepia.ceph.com' reimaging 2023-12-19T15:18:10.937 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: reset os type and version on server 2023-12-19T15:18:10.937 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-12-19T15:18:10.958 INFO:teuthology.lock.ops:Start node 'smithi132.front.sepia.ceph.com' reimaging 2023-12-19T15:18:10.958 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: reset os type and version on server 2023-12-19T15:18:10.958 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-12-19T15:18:11.074 DEBUG:teuthology.provision.fog.smithi012:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-19T15:18:11.086 DEBUG:teuthology.provision.fog.smithi132:Requesting image smithi_rhel_8.4 (ID 7) 2023-12-19T15:18:11.123 INFO:teuthology.provision.fog.smithi012:Scheduling deploy of rhel 8.4 2023-12-19T15:18:11.136 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of rhel 8.4 2023-12-19T15:18:11.401 INFO:teuthology.orchestra.console.smithi012:Power off 2023-12-19T15:18:11.401 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T15:18:11.422 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2023-12-19T15:18:11.423 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:18:11.440 INFO:teuthology.orchestra.console.smithi132:Power off 2023-12-19T15:18:11.440 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T15:18:11.448 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-12-19T15:18:11.462 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2023-12-19T15:18:11.462 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:18:11.484 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-12-19T15:18:11.549 INFO:teuthology.orchestra.console.smithi012:Power off completed 2023-12-19T15:18:11.585 INFO:teuthology.orchestra.console.smithi132:Power off completed 2023-12-19T15:18:11.650 INFO:teuthology.orchestra.console.smithi012:Power on 2023-12-19T15:18:11.650 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-19T15:18:11.675 DEBUG:teuthology.orchestra.console.smithi012:power on output: Chassis Power Control: Up/On 2023-12-19T15:18:11.676 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:18:11.686 INFO:teuthology.orchestra.console.smithi132:Power on 2023-12-19T15:18:11.687 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-19T15:18:11.698 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-12-19T15:18:11.709 DEBUG:teuthology.orchestra.console.smithi132:power on output: Chassis Power Control: Up/On 2023-12-19T15:18:11.709 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:18:11.730 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-12-19T15:18:15.701 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:18:15.732 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:18:15.810 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-12-19T15:18:15.842 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-12-19T15:18:19.816 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:18:19.843 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:18:19.926 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is on 2023-12-19T15:18:20.027 INFO:teuthology.orchestra.console.smithi012:Power on completed 2023-12-19T15:18:20.035 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2023-12-19T15:18:20.129 INFO:teuthology.provision.fog.smithi012:Waiting for deploy to finish 2023-12-19T15:18:20.135 INFO:teuthology.orchestra.console.smithi132:Power on completed 2023-12-19T15:18:20.237 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2023-12-19T15:20:51.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:20:54.294 WARNING:teuthology.provision.fog.smithi012:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-12-19T15:21:00.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:03.377 WARNING:teuthology.provision.fog.smithi012:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-12-19T15:21:06.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:09.329 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-12-19T15:21:09.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:12.437 WARNING:teuthology.provision.fog.smithi012:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-12-19T15:21:15.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:18.385 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-12-19T15:21:18.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:21.489 WARNING:teuthology.provision.fog.smithi012:[Errno None] Unable to connect to port 22 on 172.21.15.12 2023-12-19T15:21:24.389 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:27.445 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-12-19T15:21:27.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:33.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:36.534 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-12-19T15:21:42.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:21:45.621 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-12-19T15:21:51.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:22:27.526 WARNING:teuthology.provision.fog.smithi012:timed out 2023-12-19T15:22:33.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:22:33.784 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-19T15:22:33.785 DEBUG:teuthology.orchestra.run.smithi012:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-19T15:22:34.080 INFO:teuthology.provision.fog.smithi012:Node is ready 2023-12-19T15:22:34.080 DEBUG:teuthology.orchestra.run.smithi012:> hostname 2023-12-19T15:22:34.136 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012.front.sepia.ceph.com 2023-12-19T15:22:34.136 DEBUG:teuthology.orchestra.run.smithi012:> grep smithi012.front.sepia.ceph.com /etc/hosts 2023-12-19T15:22:34.194 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12 smithi012.front.sepia.ceph.com smithi012 2023-12-19T15:22:34.194 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname smithi012 2023-12-19T15:22:34.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i -e 's/smithi012.front.sepia.ceph.com/smithi012/g' /etc/hosts 2023-12-19T15:22:34.335 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i -e 's/172.21.15.12/172.21.15.12/g' /etc/hosts 2023-12-19T15:22:34.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i -e 's/smithi012.front.sepia.ceph.com/smithi012/g' /etc/hostname 2023-12-19T15:22:34.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname smithi012 2023-12-19T15:22:34.532 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2023-12-19T15:22:34.586 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8.4 (Ootpa)" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:ID="rhel" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="fedora" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8.4" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T15:22:34.587 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-19T15:22:34.588 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T15:22:34.588 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-19T15:22:34.588 INFO:teuthology.provision.fog.smithi012:Deploy complete! 2023-12-19T15:22:34.590 DEBUG:teuthology.parallel:result is None 2023-12-19T15:22:51.646 WARNING:teuthology.provision.fog.smithi132:timed out 2023-12-19T15:22:57.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:22:57.986 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-19T15:22:57.986 DEBUG:teuthology.orchestra.run.smithi132:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-19T15:22:58.253 INFO:teuthology.provision.fog.smithi132:Node is ready 2023-12-19T15:22:58.254 DEBUG:teuthology.orchestra.run.smithi132:> hostname 2023-12-19T15:22:58.269 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com 2023-12-19T15:22:58.269 DEBUG:teuthology.orchestra.run.smithi132:> grep smithi132.front.sepia.ceph.com /etc/hosts 2023-12-19T15:22:58.328 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132 2023-12-19T15:22:58.328 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-12-19T15:22:58.453 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132.front.sepia.ceph.com/smithi132/g' /etc/hosts 2023-12-19T15:22:58.478 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/172.21.15.132/172.21.15.132/g' /etc/hosts 2023-12-19T15:22:58.546 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132.front.sepia.ceph.com/smithi132/g' /etc/hostname 2023-12-19T15:22:58.614 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-12-19T15:22:58.680 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8.4 (Ootpa)" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:ID="rhel" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="fedora" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8.4" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T15:22:58.737 INFO:teuthology.orchestra.run.smithi132.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T15:22:58.738 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T15:22:58.738 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-12-19T15:22:58.738 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T15:22:58.738 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-19T15:22:58.738 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T15:22:58.738 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-19T15:22:58.738 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2023-12-19T15:22:58.739 DEBUG:teuthology.parallel:result is None 2023-12-19T15:22:58.921 INFO:teuthology.lock.ops:Checking smithi012.front.sepia.ceph.com 2023-12-19T15:22:58.922 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-19T15:22:58.943 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com 2023-12-19T15:22:58.943 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-19T15:22:58.960 INFO:teuthology.lock.ops:Updating [smithi012.front.sepia.ceph.com]: set os type and version on server 2023-12-19T15:22:58.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-12-19T15:22:58.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:22:59.030 DEBUG:teuthology.orchestra.run.smithi012:> true 2023-12-19T15:22:59.149 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2023-12-19T15:22:59.149 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2023-12-19T15:22:59.204 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2023-12-19T15:22:59.205 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2023-12-19T15:22:59.260 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T15:22:59.260 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="8.4 (Ootpa)" 2023-12-19T15:22:59.260 INFO:teuthology.orchestra.run.smithi012.stdout:ID="rhel" 2023-12-19T15:22:59.260 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="fedora" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="8.4" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el8" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout: 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T15:22:59.261 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-19T15:22:59.262 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2023-12-19T15:22:59.284 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server 2023-12-19T15:22:59.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-12-19T15:22:59.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T15:22:59.353 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-12-19T15:22:59.375 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-12-19T15:22:59.375 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-12-19T15:22:59.432 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-12-19T15:22:59.433 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8.4 (Ootpa)" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:ID="rhel" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="fedora" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8.4" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T15:22:59.491 INFO:teuthology.orchestra.run.smithi132.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T15:22:59.492 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T15:22:59.492 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-12-19T15:22:59.492 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T15:22:59.492 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-12-19T15:22:59.492 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T15:22:59.492 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-12-19T15:22:59.492 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-12-19T15:22:59.518 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T15:22:59.599 INFO:teuthology.dispatcher.supervisor:Running job 7497002 2023-12-19T15:22:59.599 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7497002 --name rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi --description fs/snaps/{begin clusters/1a3s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/workunit/snaps} -- /home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7497002/orig.config.yaml 2023-12-19T15:22:59.602 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7497002 2023-12-19T15:22:59.602 INFO:teuthology.dispatcher.supervisor:Job PID: 2950013 2023-12-19T15:22:59.602 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-19T15:24:59.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T15:26:59.870 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T15:29:00.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T15:31:00.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T15:33:00.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T15:33:00.445 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-19T15:33:00.549 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-19T15:33:00.550 INFO:teuthology.nuke:Checking targets against current locks 2023-12-19T15:33:00.606 DEBUG:teuthology.nuke:shortname: smithi012 2023-12-19T15:33:00.606 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T15:33:00.631 DEBUG:teuthology.nuke:shortname: smithi132 2023-12-19T15:33:00.631 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T15:33:00.632 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7497002', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 15:18:09.956157', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL9cQODLKmiCvnLxF5p4Dzym0zLDQ6AOQueJc5lkcjjIGi4/gyRSIEmu+3Py0rGMCKpTZ8Kxxq35I7jOIWYfE9s='} 2023-12-19T15:33:00.656 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_12:42:58-fs-rishabh-mdscap-pacific-testing-default-smithi/7497002', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 15:18:09.957230', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN6Y/qCLQsV5pLJkgfxsa6/ZnEINmLd0CmlHDAmQThPxLkarYage4mUwWsBkdldFdrY6hlcIM6EyMTy01HCfg+4='} 2023-12-19T15:33:00.687 INFO:teuthology.orchestra.console.smithi012:Power off 2023-12-19T15:33:00.687 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T15:33:00.715 DEBUG:teuthology.orchestra.console.smithi012:power off output: Chassis Power Control: Down/Off 2023-12-19T15:33:00.715 DEBUG:teuthology.orchestra.console.smithi012:pexpect command: ipmitool -H smithi012.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:33:00.733 INFO:teuthology.orchestra.console.smithi132:Power off 2023-12-19T15:33:00.734 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T15:33:00.742 DEBUG:teuthology.orchestra.console.smithi012:check power output: Chassis Power is off 2023-12-19T15:33:00.755 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2023-12-19T15:33:00.755 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T15:33:00.776 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2023-12-19T15:33:00.844 INFO:teuthology.orchestra.console.smithi012:Power off completed 2023-12-19T15:33:00.877 INFO:teuthology.orchestra.console.smithi132:Power off completed 2023-12-19T15:33:00.993 INFO:teuthology.lock.ops:unlocked: smithi012.front.sepia.ceph.com 2023-12-19T15:33:01.025 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com