2024-01-08T07:13:48.200 INFO:root:teuthology version: 0.0.1.dev228+g01dd171 2024-01-08T07:13:48.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:13:48.406 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1c0c3c0940>, signals=[15]) 2024-01-08T07:13:48.406 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2024-01-08T07:13:48.407 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2024-01-08T07:13:48.407 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-01-08T07:13:48.432 INFO:teuthology.lock.ops:Start node 'smithi175.front.sepia.ceph.com' reimaging 2024-01-08T07:13:48.432 INFO:teuthology.lock.ops:Updating [smithi175.front.sepia.ceph.com]: reset os type and version on server 2024-01-08T07:13:48.432 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-01-08T07:13:48.555 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-08T07:13:48.570 DEBUG:teuthology.provision.fog.smithi175:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-08T07:13:48.606 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of rhel 8.6 2024-01-08T07:13:48.609 INFO:teuthology.provision.fog.smithi175:Scheduling deploy of rhel 8.6 2024-01-08T07:13:48.908 INFO:teuthology.orchestra.console.smithi175:Power off 2024-01-08T07:13:48.909 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-08T07:13:48.939 INFO:teuthology.orchestra.console.smithi112:Power off 2024-01-08T07:13:48.940 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-08T07:13:48.975 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2024-01-08T07:13:48.976 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:13:49.028 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-01-08T07:13:49.028 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:13:49.076 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2024-01-08T07:13:49.113 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-01-08T07:13:49.177 INFO:teuthology.orchestra.console.smithi175:Power off completed 2024-01-08T07:13:49.214 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-01-08T07:13:49.278 INFO:teuthology.orchestra.console.smithi175:Power on 2024-01-08T07:13:49.279 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-08T07:13:49.315 INFO:teuthology.orchestra.console.smithi112:Power on 2024-01-08T07:13:49.315 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-08T07:13:49.351 DEBUG:teuthology.orchestra.console.smithi175:power on output: Chassis Power Control: Up/On 2024-01-08T07:13:49.351 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:13:49.387 DEBUG:teuthology.orchestra.console.smithi112:power on output: Chassis Power Control: Up/On 2024-01-08T07:13:49.387 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:13:49.435 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2024-01-08T07:13:49.458 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-01-08T07:13:53.440 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:13:53.481 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:13:53.584 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is on 2024-01-08T07:13:53.624 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-01-08T07:13:53.685 INFO:teuthology.orchestra.console.smithi175:Power on completed 2024-01-08T07:13:53.789 INFO:teuthology.provision.fog.smithi175:Waiting for deploy to finish 2024-01-08T07:13:57.629 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:13:57.775 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-01-08T07:13:57.878 INFO:teuthology.orchestra.console.smithi112:Power on completed 2024-01-08T07:13:57.980 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2024-01-08T07:16:39.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:16:42.780 WARNING:teuthology.provision.fog.smithi175:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-01-08T07:16:43.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:16:46.941 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-08T07:16:48.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:16:51.838 WARNING:teuthology.provision.fog.smithi175:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-01-08T07:16:52.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:16:55.996 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-08T07:16:57.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:17:00.893 WARNING:teuthology.provision.fog.smithi175:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-01-08T07:17:01.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:17:05.052 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-08T07:17:06.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:17:09.952 WARNING:teuthology.provision.fog.smithi175:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-01-08T07:17:11.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:17:14.108 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-08T07:17:15.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:17:19.038 WARNING:teuthology.provision.fog.smithi175:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-01-08T07:17:20.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:17:23.200 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-01-08T07:17:25.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:17:28.092 WARNING:teuthology.provision.fog.smithi175:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-01-08T07:17:29.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:17:34.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:18:29.260 WARNING:teuthology.provision.fog.smithi112:timed out 2024-01-08T07:18:34.102 WARNING:teuthology.provision.fog.smithi175:timed out 2024-01-08T07:18:35.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:18:35.488 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-08T07:18:35.488 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-08T07:18:35.741 INFO:teuthology.provision.fog.smithi112:Node is ready 2024-01-08T07:18:35.741 DEBUG:teuthology.orchestra.run.smithi112:> hostname 2024-01-08T07:18:35.801 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2024-01-08T07:18:35.802 DEBUG:teuthology.orchestra.run.smithi112:> grep smithi112.front.sepia.ceph.com /etc/hosts 2024-01-08T07:18:35.861 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2024-01-08T07:18:35.861 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-01-08T07:18:35.975 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hosts 2024-01-08T07:18:36.006 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/172.21.15.112/172.21.15.112/g' /etc/hosts 2024-01-08T07:18:36.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hostname 2024-01-08T07:18:36.144 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2024-01-08T07:18:36.216 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-01-08T07:18:36.272 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Red Hat Enterprise Linux" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8.6 (Ootpa)" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:ID="rhel" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="fedora" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8.6" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.redhat.com/" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-08T07:18:36.273 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-08T07:18:36.274 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-08T07:18:36.274 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2024-01-08T07:18:36.275 DEBUG:teuthology.parallel:result is None 2024-01-08T07:18:40.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:18:40.360 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-08T07:18:40.360 DEBUG:teuthology.orchestra.run.smithi175:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-08T07:18:40.617 INFO:teuthology.provision.fog.smithi175:Node is ready 2024-01-08T07:18:40.618 DEBUG:teuthology.orchestra.run.smithi175:> hostname 2024-01-08T07:18:40.635 INFO:teuthology.orchestra.run.smithi175.stdout:smithi175 2024-01-08T07:18:40.636 DEBUG:teuthology.orchestra.run.smithi175:> grep smithi175 /etc/hosts 2024-01-08T07:18:40.694 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.15.175 smithi175.front.sepia.ceph.com smithi175 2024-01-08T07:18:40.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname smithi175 2024-01-08T07:18:40.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/smithi175/smithi175/g' /etc/hosts 2024-01-08T07:18:40.844 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/172.21.15.175/172.21.15.175/g' /etc/hosts 2024-01-08T07:18:40.913 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i -e 's/smithi175/smithi175/g' /etc/hostname 2024-01-08T07:18:40.983 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname smithi175 2024-01-08T07:18:41.050 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-01-08T07:18:41.108 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Red Hat Enterprise Linux" 2024-01-08T07:18:41.108 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="8.6 (Ootpa)" 2024-01-08T07:18:41.108 INFO:teuthology.orchestra.run.smithi175.stdout:ID="rhel" 2024-01-08T07:18:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="fedora" 2024-01-08T07:18:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="8.6" 2024-01-08T07:18:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el8" 2024-01-08T07:18:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-08T07:18:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-01-08T07:18:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-08T07:18:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.redhat.com/" 2024-01-08T07:18:41.109 INFO:teuthology.orchestra.run.smithi175.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-08T07:18:41.110 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-08T07:18:41.110 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-01-08T07:18:41.110 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-08T07:18:41.110 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-08T07:18:41.110 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-08T07:18:41.110 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-08T07:18:41.110 INFO:teuthology.provision.fog.smithi175:Deploy complete! 2024-01-08T07:18:41.111 DEBUG:teuthology.parallel:result is None 2024-01-08T07:18:41.340 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2024-01-08T07:18:41.340 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-08T07:18:41.369 INFO:teuthology.lock.ops:Checking smithi175.front.sepia.ceph.com 2024-01-08T07:18:41.369 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-08T07:18:41.386 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2024-01-08T07:18:41.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-01-08T07:18:41.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:18:41.461 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-01-08T07:18:41.534 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-01-08T07:18:41.534 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-01-08T07:18:41.591 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-01-08T07:18:41.592 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-01-08T07:18:41.650 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Red Hat Enterprise Linux" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8.6 (Ootpa)" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:ID="rhel" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="fedora" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8.6" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.redhat.com/" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-08T07:18:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-01-08T07:18:41.652 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-08T07:18:41.652 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-08T07:18:41.652 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-08T07:18:41.652 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-08T07:18:41.652 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-01-08T07:18:41.677 INFO:teuthology.lock.ops:Updating [smithi175.front.sepia.ceph.com]: set os type and version on server 2024-01-08T07:18:41.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-01-08T07:18:41.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:18:41.751 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-01-08T07:18:41.821 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-01-08T07:18:41.821 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-01-08T07:18:41.880 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-01-08T07:18:41.881 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-01-08T07:18:41.939 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="Red Hat Enterprise Linux" 2024-01-08T07:18:41.939 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="8.6 (Ootpa)" 2024-01-08T07:18:41.939 INFO:teuthology.orchestra.run.smithi175.stdout:ID="rhel" 2024-01-08T07:18:41.939 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="fedora" 2024-01-08T07:18:41.939 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="8.6" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el8" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://www.redhat.com/" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-08T07:18:41.940 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-08T07:18:41.941 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-01-08T07:18:41.960 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:18:42.068 INFO:teuthology.dispatcher.supervisor:Running job 7508902 2024-01-08T07:18:42.068 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508902 --name vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} -- /home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508902/orig.config.yaml 2024-01-08T07:18:42.076 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508902 2024-01-08T07:18:42.076 INFO:teuthology.dispatcher.supervisor:Job PID: 4190940 2024-01-08T07:18:42.076 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-01-08T07:20:42.177 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:22:42.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:24:42.516 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:26:42.692 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:28:42.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:30:43.024 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:30:43.090 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-01-08T07:30:43.195 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2024-01-08T07:30:43.196 INFO:teuthology.nuke:Checking targets against current locks 2024-01-08T07:30:43.236 DEBUG:teuthology.nuke:shortname: smithi112 2024-01-08T07:30:43.236 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-08T07:30:43.256 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508902', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-08 07:13:47.001323', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNN7xtWB13AzoyI35OJbLUC8duIGHQEZeUkJrDZgrHwi3pbQJrtgxror7LIctmuXSACyEwS0pbNIpMbhtmEuL04='} 2024-01-08T07:30:43.260 DEBUG:teuthology.nuke:shortname: smithi175 2024-01-08T07:30:43.260 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-08T07:30:43.282 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508902', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-08 07:13:47.003383', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKQtgxX1R56PNqgNPfhdFF0TtRTbrb8+WMUy+enOltPTOzFpX3Q1nxgdmDBiTCe7ep8XfVVMWls+d7RMY13IPqE='} 2024-01-08T07:30:43.295 INFO:teuthology.orchestra.console.smithi112:Power off 2024-01-08T07:30:43.296 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-08T07:30:43.317 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-01-08T07:30:43.317 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:30:43.331 INFO:teuthology.orchestra.console.smithi175:Power off 2024-01-08T07:30:43.331 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-08T07:30:43.340 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-01-08T07:30:43.352 DEBUG:teuthology.orchestra.console.smithi175:power off output: Chassis Power Control: Down/Off 2024-01-08T07:30:43.353 DEBUG:teuthology.orchestra.console.smithi175:pexpect command: ipmitool -H smithi175.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:30:43.378 DEBUG:teuthology.orchestra.console.smithi175:check power output: Chassis Power is off 2024-01-08T07:30:43.440 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-01-08T07:30:43.479 INFO:teuthology.orchestra.console.smithi175:Power off completed 2024-01-08T07:30:43.590 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2024-01-08T07:30:43.622 INFO:teuthology.lock.ops:unlocked: smithi175.front.sepia.ceph.com