2023-07-21T06:45:35.280 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T06:45:35.281 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T06:45:35.282 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T06:45:35.439 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7c4d687f70>, signals=[15]) 2023-07-21T06:45:35.440 INFO:teuthology.lock.ops:Start node 'smithi050.front.sepia.ceph.com' reimaging 2023-07-21T06:45:35.441 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T06:45:35.441 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-07-21T06:45:35.465 INFO:teuthology.lock.ops:Start node 'smithi112.front.sepia.ceph.com' reimaging 2023-07-21T06:45:35.465 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T06:45:35.466 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-07-21T06:45:35.467 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /host {"name": "smithi050"} 2023-07-21T06:45:35.489 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /host {"name": "smithi112"} 2023-07-21T06:45:35.541 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-21T06:45:35.566 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-21T06:45:35.581 DEBUG:teuthology.provision.fog.smithi050:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-21T06:45:35.581 DEBUG:teuthology.provision.fog.smithi050:FOG request: PUT /host/155 {"imageID": 5} 2023-07-21T06:45:35.602 DEBUG:teuthology.provision.fog.smithi112:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-21T06:45:35.602 DEBUG:teuthology.provision.fog.smithi112:FOG request: PUT /host/217 {"imageID": 5} 2023-07-21T06:45:35.626 INFO:teuthology.provision.fog.smithi050:Scheduling deploy of rhel 8.6 2023-07-21T06:45:35.626 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:45:35.656 INFO:teuthology.provision.fog.smithi112:Scheduling deploy of rhel 8.6 2023-07-21T06:45:35.657 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:45:35.705 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T06:45:35.735 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T06:45:35.744 DEBUG:teuthology.provision.fog.smithi050:FOG request: POST /host/155/task {"taskTypeID": 1} 2023-07-21T06:45:35.773 DEBUG:teuthology.provision.fog.smithi112:FOG request: POST /host/217/task {"taskTypeID": 1} 2023-07-21T06:45:35.802 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:45:35.827 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:45:35.921 INFO:teuthology.orchestra.console:Power off smithi050 2023-07-21T06:45:35.922 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T06:45:35.939 INFO:teuthology.orchestra.console:Power off smithi112 2023-07-21T06:45:35.940 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T06:45:35.950 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:35.962 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:39.977 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:39.987 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:45.088 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:45.098 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:50.201 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:50.212 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:55.314 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:45:55.322 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:00.425 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:00.434 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:00.646 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2023-07-21T06:46:00.747 INFO:teuthology.orchestra.console:Power on smithi112 2023-07-21T06:46:00.747 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T06:46:00.770 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:04.795 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:05.539 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:09.906 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:10.652 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:15.019 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:15.764 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:20.130 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:20.341 INFO:teuthology.orchestra.console:Power on for smithi112 completed 2023-07-21T06:46:20.443 INFO:teuthology.provision.fog.smithi112:Waiting for deploy to finish 2023-07-21T06:46:20.443 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:46:24.339 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:29.452 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:34.564 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:39.422 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:46:54.433 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:54.644 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2023-07-21T06:46:54.746 INFO:teuthology.orchestra.console:Power on smithi050 2023-07-21T06:46:54.746 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T06:46:54.769 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:46:58.796 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:47:02.908 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:47:08.020 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:47:09.498 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:47:14.497 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:47:19.609 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:47:24.721 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:47:24.932 INFO:teuthology.orchestra.console:Power on for smithi050 completed 2023-07-21T06:47:25.033 INFO:teuthology.provision.fog.smithi050:Waiting for deploy to finish 2023-07-21T06:47:25.034 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:47:30.041 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:47:50.120 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:47:50.122 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:48:05.205 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:48:05.206 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:48:30.297 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:48:35.305 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:48:55.372 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:48:55.374 DEBUG:teuthology.provision.fog.smithi112:FOG request: GET /task/active None 2023-07-21T06:49:05.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:49:13.520 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-21T06:49:19.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:49:25.444 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:49:27.600 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-21T06:49:33.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:49:40.520 DEBUG:teuthology.provision.fog.smithi050:FOG request: GET /task/active None 2023-07-21T06:49:40.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:49:41.680 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-21T06:49:43.664 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-07-21T06:49:47.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:49:52.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:49:55.760 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-07-21T06:50:01.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:50:04.816 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-07-21T06:50:10.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:50:15.825 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-21T06:50:18.900 WARNING:teuthology.provision.fog.smithi050:[Errno None] Unable to connect to port 22 on 172.21.15.50 2023-07-21T06:50:21.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:50:24.880 WARNING:teuthology.provision.fog.smithi112:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-07-21T06:50:24.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:50:30.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:50:36.143 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T06:50:36.144 DEBUG:teuthology.orchestra.run.smithi112:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T06:50:36.416 INFO:teuthology.provision.fog.smithi112:Node is ready 2023-07-21T06:50:36.417 DEBUG:teuthology.orchestra.run.smithi112:> hostname 2023-07-21T06:50:36.434 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112.front.sepia.ceph.com 2023-07-21T06:50:36.434 DEBUG:teuthology.orchestra.run.smithi112:> grep smithi112.front.sepia.ceph.com /etc/hosts 2023-07-21T06:50:36.492 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112 smithi112.front.sepia.ceph.com smithi112 2023-07-21T06:50:36.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2023-07-21T06:50:36.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hosts 2023-07-21T06:50:36.634 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/172.21.15.112/172.21.15.112/g' /etc/hosts 2023-07-21T06:50:36.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i -e 's/smithi112.front.sepia.ceph.com/smithi112/g' /etc/hostname 2023-07-21T06:50:36.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname smithi112 2023-07-21T06:50:36.834 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-07-21T06:50:36.891 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T06:50:36.891 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T06:50:36.891 INFO:teuthology.orchestra.run.smithi112.stdout:ID="rhel" 2023-07-21T06:50:36.891 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="fedora" 2023-07-21T06:50:36.892 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8.6" 2023-07-21T06:50:36.892 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2023-07-21T06:50:36.892 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T06:50:36.892 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2023-07-21T06:50:36.892 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T06:50:36.893 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T06:50:36.893 INFO:teuthology.orchestra.run.smithi112.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T06:50:36.893 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T06:50:36.893 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-07-21T06:50:36.893 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T06:50:36.893 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T06:50:36.893 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T06:50:36.894 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T06:50:36.894 INFO:teuthology.provision.fog.smithi112:Deploy complete! 2023-07-21T06:50:36.895 DEBUG:teuthology.parallel:result is None 2023-07-21T06:51:29.956 WARNING:teuthology.provision.fog.smithi050:timed out 2023-07-21T06:51:35.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:51:36.287 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T06:51:36.288 DEBUG:teuthology.orchestra.run.smithi050:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T06:51:36.567 INFO:teuthology.provision.fog.smithi050:Node is ready 2023-07-21T06:51:36.567 DEBUG:teuthology.orchestra.run.smithi050:> hostname 2023-07-21T06:51:36.586 INFO:teuthology.orchestra.run.smithi050.stdout:smithi050.front.sepia.ceph.com 2023-07-21T06:51:36.586 DEBUG:teuthology.orchestra.run.smithi050:> grep smithi050.front.sepia.ceph.com /etc/hosts 2023-07-21T06:51:36.646 INFO:teuthology.orchestra.run.smithi050.stdout:172.21.15.50 smithi050.front.sepia.ceph.com smithi050 2023-07-21T06:51:36.646 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2023-07-21T06:51:36.777 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hosts 2023-07-21T06:51:36.807 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/172.21.15.50/172.21.15.50/g' /etc/hosts 2023-07-21T06:51:36.877 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i -e 's/smithi050.front.sepia.ceph.com/smithi050/g' /etc/hostname 2023-07-21T06:51:36.946 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname smithi050 2023-07-21T06:51:37.015 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2023-07-21T06:51:37.072 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T06:51:37.072 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T06:51:37.072 INFO:teuthology.orchestra.run.smithi050.stdout:ID="rhel" 2023-07-21T06:51:37.073 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="fedora" 2023-07-21T06:51:37.073 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8.6" 2023-07-21T06:51:37.073 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8" 2023-07-21T06:51:37.073 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T06:51:37.073 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2023-07-21T06:51:37.073 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T06:51:37.073 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T06:51:37.074 INFO:teuthology.orchestra.run.smithi050.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T06:51:37.074 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T06:51:37.074 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-07-21T06:51:37.074 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T06:51:37.074 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T06:51:37.074 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T06:51:37.074 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T06:51:37.075 INFO:teuthology.provision.fog.smithi050:Deploy complete! 2023-07-21T06:51:37.075 DEBUG:teuthology.parallel:result is None 2023-07-21T06:51:42.236 INFO:teuthology.lock.ops:Checking smithi050.front.sepia.ceph.com 2023-07-21T06:51:42.237 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T06:51:47.296 INFO:teuthology.lock.ops:Checking smithi112.front.sepia.ceph.com 2023-07-21T06:51:47.298 INFO:teuthology.lock.ops:Updating [smithi050.front.sepia.ceph.com]: set os type and version on server 2023-07-21T06:51:47.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-07-21T06:51:47.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:51:52.372 DEBUG:teuthology.orchestra.run.smithi050:> true 2023-07-21T06:51:52.501 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2023-07-21T06:51:52.502 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2023-07-21T06:51:52.559 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2023-07-21T06:51:52.560 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2023-07-21T06:51:52.617 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T06:51:52.618 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T06:51:52.618 INFO:teuthology.orchestra.run.smithi050.stdout:ID="rhel" 2023-07-21T06:51:52.618 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="fedora" 2023-07-21T06:51:52.618 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="8.6" 2023-07-21T06:51:52.618 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el8" 2023-07-21T06:51:52.618 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T06:51:52.619 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2023-07-21T06:51:52.619 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T06:51:52.619 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T06:51:52.619 INFO:teuthology.orchestra.run.smithi050.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T06:51:52.619 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T06:51:52.619 INFO:teuthology.orchestra.run.smithi050.stdout: 2023-07-21T06:51:52.619 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T06:51:52.619 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T06:51:52.620 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T06:51:52.620 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T06:51:52.620 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2023-07-21T06:51:57.642 INFO:teuthology.lock.ops:Updating [smithi112.front.sepia.ceph.com]: set os type and version on server 2023-07-21T06:51:57.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-07-21T06:51:57.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:52:02.711 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-07-21T06:52:02.846 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-07-21T06:52:02.847 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2023-07-21T06:52:02.903 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2023-07-21T06:52:02.903 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-07-21T06:52:02.960 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T06:52:02.960 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T06:52:02.960 INFO:teuthology.orchestra.run.smithi112.stdout:ID="rhel" 2023-07-21T06:52:02.960 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="fedora" 2023-07-21T06:52:02.960 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="8.6" 2023-07-21T06:52:02.961 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el8" 2023-07-21T06:52:02.961 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T06:52:02.961 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2023-07-21T06:52:02.961 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T06:52:02.961 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T06:52:02.961 INFO:teuthology.orchestra.run.smithi112.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T06:52:02.961 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T06:52:02.962 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-07-21T06:52:02.962 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T06:52:02.962 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T06:52:02.962 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T06:52:02.962 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T06:52:02.963 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-07-21T06:52:08.000 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T06:52:08.001 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T06:52:13.105 INFO:teuthology.dispatcher.supervisor:Running job 7346379 2023-07-21T06:52:13.105 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346379 --name vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/workunit/quota} -- /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346379/orig.config.yaml 2023-07-21T06:52:13.108 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346379 2023-07-21T06:52:13.109 INFO:teuthology.dispatcher.supervisor:Job PID: 7087 2023-07-21T06:52:13.109 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-21T06:54:13.208 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T06:54:13.209 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T06:56:18.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T06:56:18.401 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T06:58:18.585 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T06:58:18.585 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:00:23.093 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:00:23.094 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:02:23.393 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:02:23.394 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:04:28.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:04:28.552 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:06:30.897 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:06:30.898 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:08:36.058 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:08:36.059 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:10:41.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:10:41.225 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:12:46.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:12:46.390 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:14:51.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:14:51.611 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:16:56.783 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:16:56.784 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:19:01.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:19:01.938 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:21:07.113 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:21:07.114 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:23:08.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:23:08.654 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:25:13.845 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:25:13.846 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:27:14.017 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T07:27:14.018 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T07:27:18.815 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-21T07:27:29.018 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-21T07:27:29.019 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T07:27:43.787 DEBUG:teuthology.nuke:shortname: smithi050 2023-07-21T07:27:43.787 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T07:27:43.801 DEBUG:teuthology.nuke:shortname: smithi112 2023-07-21T07:27:43.802 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T07:27:43.812 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346379', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 06:45:34.468100', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T07:27:48.818 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346379', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 06:45:34.466595', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T07:27:59.756 INFO:teuthology.orchestra.console:Power off smithi112 2023-07-21T07:27:59.757 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T07:27:59.770 INFO:teuthology.orchestra.console:Power off smithi050 2023-07-21T07:27:59.771 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T07:28:04.795 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:28:09.808 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:28:14.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:28:14.926 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:28:14.938 INFO:teuthology.orchestra.console:Power off for smithi112 completed 2023-07-21T07:28:25.072 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:28:25.109 INFO:teuthology.lock.ops:unlocked: smithi112.front.sepia.ceph.com 2023-07-21T07:28:30.185 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:28:35.296 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T07:28:35.509 INFO:teuthology.orchestra.console:Power off for smithi050 completed 2023-07-21T07:28:45.671 INFO:teuthology.lock.ops:unlocked: smithi050.front.sepia.ceph.com