2023-06-28T10:19:22.037 INFO:root:teuthology version: 0.0.1.dev132+gaa9345a 2023-06-28T10:19:22.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:19:22.039 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:19:31.810 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f71deb7e8b0>, signals=[15]) 2023-06-28T10:19:31.812 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging 2023-06-28T10:19:31.813 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server 2023-06-28T10:19:31.814 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-06-28T10:19:31.839 INFO:teuthology.lock.ops:Start node 'smithi071.front.sepia.ceph.com' reimaging 2023-06-28T10:19:31.840 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: reset os type and version on server 2023-06-28T10:19:31.840 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2023-06-28T10:19:31.841 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /host {"name": "smithi062"} 2023-06-28T10:19:31.864 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /host {"name": "smithi071"} 2023-06-28T10:19:31.910 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-06-28T10:19:31.929 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-06-28T10:19:31.952 DEBUG:teuthology.provision.fog.smithi062:Requesting image smithi_rhel_8.6 (ID 5) 2023-06-28T10:19:31.952 DEBUG:teuthology.provision.fog.smithi062:FOG request: PUT /host/167 {"imageID": 5} 2023-06-28T10:19:31.961 DEBUG:teuthology.provision.fog.smithi071:Requesting image smithi_rhel_8.6 (ID 5) 2023-06-28T10:19:31.961 DEBUG:teuthology.provision.fog.smithi071:FOG request: PUT /host/176 {"imageID": 5} 2023-06-28T10:19:32.010 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of rhel 8.6 2023-06-28T10:19:32.010 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:19:32.014 INFO:teuthology.provision.fog.smithi071:Scheduling deploy of rhel 8.6 2023-06-28T10:19:32.014 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:19:32.080 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /tasktype {"name": "deploy"} 2023-06-28T10:19:32.082 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /tasktype {"name": "deploy"} 2023-06-28T10:19:32.114 DEBUG:teuthology.provision.fog.smithi062:FOG request: POST /host/167/task {"taskTypeID": 1} 2023-06-28T10:19:32.118 DEBUG:teuthology.provision.fog.smithi071:FOG request: POST /host/176/task {"taskTypeID": 1} 2023-06-28T10:19:32.172 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:19:32.174 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:19:32.279 INFO:teuthology.orchestra.console:Power off smithi062 2023-06-28T10:19:32.280 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T10:19:32.295 INFO:teuthology.orchestra.console:Power off smithi071 2023-06-28T10:19:32.296 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T10:19:32.305 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:32.316 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:36.332 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:36.340 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:41.444 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:41.452 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:46.556 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:46.564 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:51.668 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:51.676 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:51.878 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2023-06-28T10:19:51.980 INFO:teuthology.orchestra.console:Power on smithi062 2023-06-28T10:19:51.980 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-28T10:19:56.789 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:19:57.004 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:01.030 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:01.899 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:06.142 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:07.011 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:11.253 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:11.463 INFO:teuthology.orchestra.console:Power on for smithi062 completed 2023-06-28T10:20:11.564 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish 2023-06-28T10:20:11.565 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:20:20.956 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:21.167 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2023-06-28T10:20:21.268 INFO:teuthology.orchestra.console:Power on smithi071 2023-06-28T10:20:21.268 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-28T10:20:21.289 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:25.312 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:20:25.523 INFO:teuthology.orchestra.console:Power on for smithi071 completed 2023-06-28T10:20:25.624 INFO:teuthology.provision.fog.smithi071:Waiting for deploy to finish 2023-06-28T10:20:25.625 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:20:36.032 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:20:45.709 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:20:56.061 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:21:05.784 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:21:16.063 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:21:20.848 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:21:31.884 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:21:36.497 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:21:51.595 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:21:51.596 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:22:11.680 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:22:16.687 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:22:31.772 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:22:36.777 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:22:51.884 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:22:51.885 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-06-28T10:23:01.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:05.040 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-06-28T10:23:11.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:16.964 DEBUG:teuthology.provision.fog.smithi071:FOG request: GET /task/active None 2023-06-28T10:23:17.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:19.120 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-06-28T10:23:20.208 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-06-28T10:23:25.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:26.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:28.176 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-06-28T10:23:29.264 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-06-28T10:23:34.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:39.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:44.189 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-06-28T10:23:47.252 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-06-28T10:23:50.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:53.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:23:56.336 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-06-28T10:24:02.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:24:10.416 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-06-28T10:24:16.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:24:21.854 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-28T10:24:21.855 DEBUG:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-28T10:24:22.152 INFO:teuthology.provision.fog.smithi062:Node is ready 2023-06-28T10:24:22.153 DEBUG:teuthology.orchestra.run.smithi062:> hostname 2023-06-28T10:24:22.169 INFO:teuthology.orchestra.run.smithi062.stdout:smithi022 2023-06-28T10:24:22.171 DEBUG:teuthology.orchestra.run.smithi062:> grep smithi022 /etc/hosts 2023-06-28T10:24:22.226 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2023-06-28T10:24:22.226 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2023-06-28T10:24:22.347 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi022/smithi062/g' /etc/hosts 2023-06-28T10:24:22.374 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/172.21.15.22/172.21.15.62/g' /etc/hosts 2023-06-28T10:24:22.441 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi022/smithi062/g' /etc/hostname 2023-06-28T10:24:22.509 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2023-06-28T10:24:22.576 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-06-28T10:24:22.633 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Red Hat Enterprise Linux" 2023-06-28T10:24:22.633 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8.6 (Ootpa)" 2023-06-28T10:24:22.633 INFO:teuthology.orchestra.run.smithi062.stdout:ID="rhel" 2023-06-28T10:24:22.633 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="fedora" 2023-06-28T10:24:22.633 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8.6" 2023-06-28T10:24:22.634 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2023-06-28T10:24:22.634 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-28T10:24:22.634 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-06-28T10:24:22.634 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-28T10:24:22.634 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.redhat.com/" 2023-06-28T10:24:22.634 INFO:teuthology.orchestra.run.smithi062.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-28T10:24:22.634 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-28T10:24:22.634 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-06-28T10:24:22.635 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-28T10:24:22.635 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-28T10:24:22.635 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-28T10:24:22.635 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-28T10:24:22.636 INFO:teuthology.provision.fog.smithi062:Deploy complete! 2023-06-28T10:24:22.637 DEBUG:teuthology.parallel:result is None 2023-06-28T10:24:39.792 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-06-28T10:24:45.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:24:53.872 WARNING:teuthology.provision.fog.smithi071:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-06-28T10:24:59.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:25:05.202 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-28T10:25:05.202 DEBUG:teuthology.orchestra.run.smithi071:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-28T10:25:05.489 INFO:teuthology.provision.fog.smithi071:Node is ready 2023-06-28T10:25:05.490 DEBUG:teuthology.orchestra.run.smithi071:> hostname 2023-06-28T10:25:05.508 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071.front.sepia.ceph.com 2023-06-28T10:25:05.509 DEBUG:teuthology.orchestra.run.smithi071:> grep smithi071.front.sepia.ceph.com /etc/hosts 2023-06-28T10:25:05.566 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71 smithi071.front.sepia.ceph.com smithi071 2023-06-28T10:25:05.567 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname smithi071 2023-06-28T10:25:05.684 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/smithi071.front.sepia.ceph.com/smithi071/g' /etc/hosts 2023-06-28T10:25:05.711 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/172.21.15.71/172.21.15.71/g' /etc/hosts 2023-06-28T10:25:05.779 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i -e 's/smithi071.front.sepia.ceph.com/smithi071/g' /etc/hostname 2023-06-28T10:25:05.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname smithi071 2023-06-28T10:25:05.913 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2023-06-28T10:25:05.968 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Red Hat Enterprise Linux" 2023-06-28T10:25:05.969 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8.6 (Ootpa)" 2023-06-28T10:25:05.969 INFO:teuthology.orchestra.run.smithi071.stdout:ID="rhel" 2023-06-28T10:25:05.969 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="fedora" 2023-06-28T10:25:05.969 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8.6" 2023-06-28T10:25:05.969 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2023-06-28T10:25:05.969 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-28T10:25:05.969 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2023-06-28T10:25:05.970 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-28T10:25:05.970 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.redhat.com/" 2023-06-28T10:25:05.970 INFO:teuthology.orchestra.run.smithi071.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-28T10:25:05.970 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-28T10:25:05.970 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-06-28T10:25:05.970 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-28T10:25:05.970 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-28T10:25:05.970 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-28T10:25:05.971 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-28T10:25:05.971 INFO:teuthology.provision.fog.smithi071:Deploy complete! 2023-06-28T10:25:05.972 DEBUG:teuthology.parallel:result is None 2023-06-28T10:25:06.128 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com 2023-06-28T10:25:06.129 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-28T10:25:06.155 INFO:teuthology.lock.ops:Checking smithi071.front.sepia.ceph.com 2023-06-28T10:25:06.155 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-28T10:25:06.181 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server 2023-06-28T10:25:06.182 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-06-28T10:25:06.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:25:06.250 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-06-28T10:25:06.386 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-06-28T10:25:06.443 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-06-28T10:25:06.444 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-06-28T10:25:06.501 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="Red Hat Enterprise Linux" 2023-06-28T10:25:06.501 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="8.6 (Ootpa)" 2023-06-28T10:25:06.501 INFO:teuthology.orchestra.run.smithi062.stdout:ID="rhel" 2023-06-28T10:25:06.502 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="fedora" 2023-06-28T10:25:06.502 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="8.6" 2023-06-28T10:25:06.502 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el8" 2023-06-28T10:25:06.502 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-28T10:25:06.502 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-06-28T10:25:06.503 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-28T10:25:06.503 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://www.redhat.com/" 2023-06-28T10:25:06.503 INFO:teuthology.orchestra.run.smithi062.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-28T10:25:06.504 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-28T10:25:06.504 INFO:teuthology.orchestra.run.smithi062.stdout: 2023-06-28T10:25:06.504 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-28T10:25:06.504 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-28T10:25:06.505 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-28T10:25:06.505 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-28T10:25:06.505 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-06-28T10:25:11.537 INFO:teuthology.lock.ops:Updating [smithi071.front.sepia.ceph.com]: set os type and version on server 2023-06-28T10:25:11.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-06-28T10:25:11.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T10:25:16.611 DEBUG:teuthology.orchestra.run.smithi071:> true 2023-06-28T10:25:16.742 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2023-06-28T10:25:16.800 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2023-06-28T10:25:16.801 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2023-06-28T10:25:16.858 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Red Hat Enterprise Linux" 2023-06-28T10:25:16.859 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8.6 (Ootpa)" 2023-06-28T10:25:16.860 INFO:teuthology.orchestra.run.smithi071.stdout:ID="rhel" 2023-06-28T10:25:16.860 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="fedora" 2023-06-28T10:25:16.861 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8.6" 2023-06-28T10:25:16.861 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2023-06-28T10:25:16.861 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-28T10:25:16.862 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2023-06-28T10:25:16.862 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-28T10:25:16.862 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.redhat.com/" 2023-06-28T10:25:16.862 INFO:teuthology.orchestra.run.smithi071.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-28T10:25:16.863 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-28T10:25:16.863 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-06-28T10:25:16.863 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-28T10:25:16.863 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-28T10:25:16.864 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-28T10:25:16.864 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-28T10:25:16.865 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2023-06-28T10:25:24.519 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:25:24.519 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:25:24.655 INFO:teuthology.dispatcher.supervisor:Running job 7319530 2023-06-28T10:25:24.656 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-27_22:58:43-fs-reef-release-distro-default-smithi/7319530 --name yuriw-2023-06-27_22:58:43-fs-reef-release-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mds 2-workunit/fs/snaps}} -- /home/teuthworker/archive/yuriw-2023-06-27_22:58:43-fs-reef-release-distro-default-smithi/7319530/orig.config.yaml 2023-06-28T10:25:24.660 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-06-27_22:58:43-fs-reef-release-distro-default-smithi/7319530 2023-06-28T10:25:24.661 INFO:teuthology.dispatcher.supervisor:Job PID: 8625 2023-06-28T10:25:24.661 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-06-28T10:27:24.761 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:27:24.762 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:29:27.964 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:29:27.965 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:31:28.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:31:28.139 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:33:33.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:33:33.322 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:35:37.729 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:35:37.730 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:37:42.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:37:42.905 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:39:45.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:39:45.894 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:41:51.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:41:51.077 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:43:56.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:43:56.212 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:45:57.177 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:45:57.178 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:47:59.867 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:47:59.868 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:50:05.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:50:05.050 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:52:08.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:52:08.358 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:54:13.544 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T10:54:13.545 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T10:54:18.610 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-06-28T10:54:25.477 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-06-28T10:54:25.477 INFO:teuthology.nuke:Checking targets against current locks 2023-06-28T10:54:25.547 DEBUG:teuthology.nuke:shortname: smithi062 2023-06-28T10:54:25.547 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T10:54:25.568 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_22:58:43-fs-reef-release-distro-default-smithi/7319530', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 10:19:20.826772', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-28T10:54:25.571 DEBUG:teuthology.nuke:shortname: smithi071 2023-06-28T10:54:25.571 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T10:54:25.591 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_22:58:43-fs-reef-release-distro-default-smithi/7319530', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 10:19:20.825302', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-28T10:54:25.618 INFO:teuthology.orchestra.console:Power off smithi062 2023-06-28T10:54:25.619 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T10:54:25.634 INFO:teuthology.orchestra.console:Power off smithi071 2023-06-28T10:54:25.635 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T10:54:25.645 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:54:25.658 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T10:54:25.771 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2023-06-28T10:54:25.782 INFO:teuthology.orchestra.console:Power off for smithi071 completed 2023-06-28T10:54:25.931 INFO:teuthology.lock.ops:unlocked: smithi062.front.sepia.ceph.com 2023-06-28T10:54:25.938 INFO:teuthology.lock.ops:unlocked: smithi071.front.sepia.ceph.com