2023-07-14T18:51:51.930 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T18:51:51.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T18:51:51.932 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T18:51:52.118 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f47097068b0>, signals=[15]) 2023-07-14T18:51:52.120 INFO:teuthology.lock.ops:Start node 'smithi098.front.sepia.ceph.com' reimaging 2023-07-14T18:51:52.120 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T18:51:52.120 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-07-14T18:51:52.148 INFO:teuthology.lock.ops:Start node 'smithi101.front.sepia.ceph.com' reimaging 2023-07-14T18:51:52.149 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T18:51:52.149 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-07-14T18:51:52.150 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /host {"name": "smithi098"} 2023-07-14T18:51:53.972 INFO:teuthology.lock.ops:Start node 'smithi139.front.sepia.ceph.com' reimaging 2023-07-14T18:51:53.973 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T18:51:53.973 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2023-07-14T18:51:53.974 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /host {"name": "smithi101"} 2023-07-14T18:51:53.998 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /host {"name": "smithi139"} 2023-07-14T18:51:54.033 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-14T18:51:54.062 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-14T18:51:54.069 DEBUG:teuthology.provision.fog.smithi098:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-14T18:51:54.069 DEBUG:teuthology.provision.fog.smithi098:FOG request: PUT /host/203 {"imageID": 5} 2023-07-14T18:51:54.072 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-14T18:51:54.098 DEBUG:teuthology.provision.fog.smithi101:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-14T18:51:54.099 DEBUG:teuthology.provision.fog.smithi101:FOG request: PUT /host/206 {"imageID": 5} 2023-07-14T18:52:04.112 INFO:teuthology.provision.fog.smithi098:Scheduling deploy of rhel 8.6 2023-07-14T18:52:04.112 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:52:04.113 DEBUG:teuthology.provision.fog.smithi139:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-14T18:52:04.113 DEBUG:teuthology.provision.fog.smithi139:FOG request: PUT /host/244 {"imageID": 5} 2023-07-14T18:52:09.816 INFO:teuthology.provision.fog.smithi139:Scheduling deploy of rhel 8.6 2023-07-14T18:52:09.817 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:52:09.819 INFO:teuthology.provision.fog.smithi101:Scheduling deploy of rhel 8.6 2023-07-14T18:52:09.819 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:52:09.870 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T18:52:09.916 DEBUG:teuthology.provision.fog.smithi098:FOG request: POST /host/203/task {"taskTypeID": 1} 2023-07-14T18:52:09.953 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T18:52:09.956 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T18:52:09.979 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:52:09.985 DEBUG:teuthology.provision.fog.smithi101:FOG request: POST /host/206/task {"taskTypeID": 1} 2023-07-14T18:52:09.986 DEBUG:teuthology.provision.fog.smithi139:FOG request: POST /host/244/task {"taskTypeID": 1} 2023-07-14T18:52:12.499 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:52:12.500 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:52:12.535 INFO:teuthology.orchestra.console:Power off smithi098 2023-07-14T18:52:12.536 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T18:52:12.560 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:12.622 INFO:teuthology.orchestra.console:Power off smithi139 2023-07-14T18:52:12.623 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T18:52:12.641 INFO:teuthology.orchestra.console:Power off smithi101 2023-07-14T18:52:12.641 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T18:52:12.652 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:12.665 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:12.682 INFO:teuthology.orchestra.console:Power off for smithi098 completed 2023-07-14T18:52:12.776 INFO:teuthology.orchestra.console:Power off for smithi139 completed 2023-07-14T18:52:12.783 INFO:teuthology.orchestra.console:Power on smithi098 2023-07-14T18:52:12.783 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T18:52:12.791 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2023-07-14T18:52:12.804 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:12.877 INFO:teuthology.orchestra.console:Power on smithi139 2023-07-14T18:52:12.878 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T18:52:12.893 INFO:teuthology.orchestra.console:Power on smithi101 2023-07-14T18:52:12.894 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T18:52:12.904 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:12.916 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:16.828 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:16.925 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:16.936 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:21.947 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:22.037 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:22.047 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T18:52:22.258 INFO:teuthology.orchestra.console:Power on for smithi101 completed 2023-07-14T18:52:22.360 INFO:teuthology.provision.fog.smithi101:Waiting for deploy to finish 2023-07-14T18:52:22.360 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:52:27.468 INFO:teuthology.orchestra.console:Power on for smithi098 completed 2023-07-14T18:52:27.469 INFO:teuthology.orchestra.console:Power on for smithi139 completed 2023-07-14T18:52:27.569 INFO:teuthology.provision.fog.smithi098:Waiting for deploy to finish 2023-07-14T18:52:27.569 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:52:27.571 INFO:teuthology.provision.fog.smithi139:Waiting for deploy to finish 2023-07-14T18:52:27.571 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:52:42.452 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:52:44.685 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:52:44.688 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:52:59.816 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:52:59.817 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:52:59.832 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:53:14.964 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:53:19.748 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:53:19.749 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:53:34.956 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:53:39.876 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:53:39.878 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:53:55.090 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:53:55.092 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:54:05.101 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:54:25.124 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:54:29.497 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:54:29.498 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:54:54.608 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:54:54.609 DEBUG:teuthology.provision.fog.smithi139:FOG request: GET /task/active None 2023-07-14T18:54:54.610 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:55:09.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:55:12.752 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-14T18:55:18.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:55:24.686 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:55:24.998 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:55:26.832 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-14T18:55:32.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:55:37.072 WARNING:teuthology.provision.fog.smithi139:[Errno None] Unable to connect to port 22 on 172.21.15.139 2023-07-14T18:55:40.068 DEBUG:teuthology.provision.fog.smithi101:FOG request: GET /task/active None 2023-07-14T18:55:40.071 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:55:40.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:55:43.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:55:48.081 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-14T18:55:54.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:55:59.094 DEBUG:teuthology.provision.fog.smithi098:FOG request: GET /task/active None 2023-07-14T18:56:04.102 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-14T18:56:04.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:56:10.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:56:15.109 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-07-14T18:56:18.160 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-14T18:56:21.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:56:26.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:56:30.223 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-07-14T18:56:33.296 WARNING:teuthology.provision.fog.smithi101:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-14T18:56:36.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:56:39.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:56:39.540 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-07-14T18:56:45.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:56:48.084 WARNING:teuthology.provision.fog.smithi139:timed out 2023-07-14T18:56:48.624 WARNING:teuthology.provision.fog.smithi098:[Errno None] Unable to connect to port 22 on 172.21.15.98 2023-07-14T18:56:54.088 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:56:59.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:57:04.424 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T18:57:04.425 DEBUG:teuthology.orchestra.run.smithi139:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T18:57:04.704 INFO:teuthology.provision.fog.smithi139:Node is ready 2023-07-14T18:57:04.704 DEBUG:teuthology.orchestra.run.smithi139:> hostname 2023-07-14T18:57:04.723 INFO:teuthology.orchestra.run.smithi139.stdout:smithi139 2023-07-14T18:57:04.723 DEBUG:teuthology.orchestra.run.smithi139:> grep smithi139 /etc/hosts 2023-07-14T18:57:04.781 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.15.139 smithi139.front.sepia.ceph.com smithi139 2023-07-14T18:57:04.781 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname smithi139 2023-07-14T18:57:04.899 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/smithi139/smithi139/g' /etc/hosts 2023-07-14T18:57:04.928 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/172.21.15.139/172.21.15.139/g' /etc/hosts 2023-07-14T18:57:04.997 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i -e 's/smithi139/smithi139/g' /etc/hostname 2023-07-14T18:57:05.065 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname smithi139 2023-07-14T18:57:05.132 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2023-07-14T18:57:05.189 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:57:05.189 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:57:05.189 INFO:teuthology.orchestra.run.smithi139.stdout:ID="rhel" 2023-07-14T18:57:05.189 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="fedora" 2023-07-14T18:57:05.189 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="8.6" 2023-07-14T18:57:05.190 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:57:05.190 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:57:05.190 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2023-07-14T18:57:05.190 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:57:05.190 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:57:05.190 INFO:teuthology.orchestra.run.smithi139.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:57:05.190 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:57:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-07-14T18:57:05.191 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:57:05.191 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:57:05.191 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:57:05.191 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:57:05.192 INFO:teuthology.provision.fog.smithi139:Deploy complete! 2023-07-14T18:57:05.193 DEBUG:teuthology.parallel:result is None 2023-07-14T18:57:35.843 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T18:57:35.843 DEBUG:teuthology.orchestra.run.smithi098:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T18:57:36.144 INFO:teuthology.provision.fog.smithi098:Node is ready 2023-07-14T18:57:36.145 DEBUG:teuthology.orchestra.run.smithi098:> hostname 2023-07-14T18:57:36.163 INFO:teuthology.orchestra.run.smithi098.stdout:smithi098.front.sepia.ceph.com 2023-07-14T18:57:36.163 DEBUG:teuthology.orchestra.run.smithi098:> grep smithi098.front.sepia.ceph.com /etc/hosts 2023-07-14T18:57:36.221 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98 smithi098.front.sepia.ceph.com smithi098 2023-07-14T18:57:36.222 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2023-07-14T18:57:36.342 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hosts 2023-07-14T18:57:36.371 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/172.21.15.98/172.21.15.98/g' /etc/hosts 2023-07-14T18:57:36.439 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i -e 's/smithi098.front.sepia.ceph.com/smithi098/g' /etc/hostname 2023-07-14T18:57:36.507 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname smithi098 2023-07-14T18:57:36.575 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-07-14T18:57:36.632 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:57:36.632 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:57:36.632 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel" 2023-07-14T18:57:36.632 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora" 2023-07-14T18:57:36.633 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6" 2023-07-14T18:57:36.633 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:57:36.633 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:57:36.633 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-07-14T18:57:36.633 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:57:36.633 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:57:36.633 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:57:36.634 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:57:36.634 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-14T18:57:36.634 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:57:36.634 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:57:36.634 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:57:36.634 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:57:36.634 INFO:teuthology.provision.fog.smithi098:Deploy complete! 2023-07-14T18:57:36.635 DEBUG:teuthology.parallel:result is None 2023-07-14T18:57:39.304 WARNING:teuthology.provision.fog.smithi101:timed out 2023-07-14T18:57:45.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:57:50.663 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T18:57:50.664 DEBUG:teuthology.orchestra.run.smithi101:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T18:57:50.952 INFO:teuthology.provision.fog.smithi101:Node is ready 2023-07-14T18:57:50.952 DEBUG:teuthology.orchestra.run.smithi101:> hostname 2023-07-14T18:57:50.970 INFO:teuthology.orchestra.run.smithi101.stdout:smithi101.front.sepia.ceph.com 2023-07-14T18:57:50.971 DEBUG:teuthology.orchestra.run.smithi101:> grep smithi101.front.sepia.ceph.com /etc/hosts 2023-07-14T18:57:51.029 INFO:teuthology.orchestra.run.smithi101.stdout:172.21.15.101 smithi101.front.sepia.ceph.com smithi101 2023-07-14T18:57:51.029 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname smithi101 2023-07-14T18:57:51.148 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/smithi101.front.sepia.ceph.com/smithi101/g' /etc/hosts 2023-07-14T18:57:51.176 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/172.21.15.101/172.21.15.101/g' /etc/hosts 2023-07-14T18:57:51.244 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i -e 's/smithi101.front.sepia.ceph.com/smithi101/g' /etc/hostname 2023-07-14T18:57:51.313 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname smithi101 2023-07-14T18:57:51.381 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-07-14T18:57:51.437 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:57:51.438 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:57:51.438 INFO:teuthology.orchestra.run.smithi101.stdout:ID="rhel" 2023-07-14T18:57:51.439 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="fedora" 2023-07-14T18:57:51.439 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8.6" 2023-07-14T18:57:51.439 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:57:51.439 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:57:51.440 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-07-14T18:57:51.440 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:57:51.440 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:57:51.440 INFO:teuthology.orchestra.run.smithi101.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:57:51.441 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:57:51.441 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-14T18:57:51.441 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:57:51.441 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:57:51.442 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:57:51.442 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:57:51.442 INFO:teuthology.provision.fog.smithi101:Deploy complete! 2023-07-14T18:57:51.443 DEBUG:teuthology.parallel:result is None 2023-07-14T18:58:01.641 INFO:teuthology.lock.ops:Checking smithi098.front.sepia.ceph.com 2023-07-14T18:58:01.642 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-14T18:58:06.667 INFO:teuthology.lock.ops:Checking smithi101.front.sepia.ceph.com 2023-07-14T18:58:06.667 INFO:teuthology.lock.ops:Checking smithi139.front.sepia.ceph.com 2023-07-14T18:58:06.670 INFO:teuthology.lock.ops:Updating [smithi098.front.sepia.ceph.com]: set os type and version on server 2023-07-14T18:58:06.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-07-14T18:58:06.672 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:58:06.740 DEBUG:teuthology.orchestra.run.smithi098:> true 2023-07-14T18:58:06.874 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2023-07-14T18:58:06.875 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2023-07-14T18:58:06.931 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2023-07-14T18:58:06.932 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2023-07-14T18:58:06.988 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:58:06.988 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:58:06.989 INFO:teuthology.orchestra.run.smithi098.stdout:ID="rhel" 2023-07-14T18:58:06.989 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="fedora" 2023-07-14T18:58:06.989 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="8.6" 2023-07-14T18:58:06.989 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:58:06.989 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:58:06.989 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2023-07-14T18:58:06.989 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:58:06.990 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:58:06.990 INFO:teuthology.orchestra.run.smithi098.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:58:06.990 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:58:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: 2023-07-14T18:58:06.990 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:58:06.990 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:58:06.990 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:58:06.991 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:58:06.991 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2023-07-14T18:58:12.024 INFO:teuthology.lock.ops:Updating [smithi101.front.sepia.ceph.com]: set os type and version on server 2023-07-14T18:58:12.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-14T18:58:12.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:58:17.098 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-07-14T18:58:17.230 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-14T18:58:17.230 DEBUG:teuthology.orchestra.run.smithi101:> uname -m 2023-07-14T18:58:17.287 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2023-07-14T18:58:17.287 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-07-14T18:58:17.344 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:58:17.344 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:58:17.344 INFO:teuthology.orchestra.run.smithi101.stdout:ID="rhel" 2023-07-14T18:58:17.344 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="fedora" 2023-07-14T18:58:17.345 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8.6" 2023-07-14T18:58:17.345 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:58:17.345 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:58:17.345 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-07-14T18:58:17.345 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:58:17.345 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:58:17.345 INFO:teuthology.orchestra.run.smithi101.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:58:17.345 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:58:17.346 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-14T18:58:17.346 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:58:17.346 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:58:17.346 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:58:17.346 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:58:17.347 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-07-14T18:58:17.373 INFO:teuthology.lock.ops:Updating [smithi139.front.sepia.ceph.com]: set os type and version on server 2023-07-14T18:58:17.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-07-14T18:58:17.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T18:58:17.442 DEBUG:teuthology.orchestra.run.smithi139:> true 2023-07-14T18:58:17.578 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2023-07-14T18:58:17.578 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2023-07-14T18:58:17.636 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2023-07-14T18:58:17.636 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2023-07-14T18:58:17.694 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T18:58:17.694 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T18:58:17.694 INFO:teuthology.orchestra.run.smithi139.stdout:ID="rhel" 2023-07-14T18:58:17.695 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="fedora" 2023-07-14T18:58:17.695 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="8.6" 2023-07-14T18:58:17.695 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el8" 2023-07-14T18:58:17.696 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T18:58:17.696 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2023-07-14T18:58:17.696 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T18:58:17.696 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T18:58:17.697 INFO:teuthology.orchestra.run.smithi139.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T18:58:17.697 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T18:58:17.697 INFO:teuthology.orchestra.run.smithi139.stdout: 2023-07-14T18:58:17.697 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T18:58:17.698 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T18:58:17.698 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T18:58:17.698 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T18:58:17.699 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2023-07-14T18:58:17.724 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T18:58:17.725 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T18:58:17.843 INFO:teuthology.dispatcher.supervisor:Running job 7338055 2023-07-14T18:58:17.844 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338055 --name pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/ffsb}} -- /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338055/orig.config.yaml 2023-07-14T18:58:17.847 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338055 2023-07-14T18:58:17.848 INFO:teuthology.dispatcher.supervisor:Job PID: 5555 2023-07-14T18:58:17.848 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-14T19:00:17.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:00:17.949 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:02:23.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:02:23.133 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:04:25.397 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:04:25.398 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:06:30.581 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:06:30.582 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:08:32.105 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:08:32.105 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:10:37.294 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:10:37.295 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:12:42.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:12:42.474 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:14:46.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:14:46.710 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:16:49.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:16:49.334 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:18:54.277 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:18:54.278 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:20:59.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:20:59.450 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:23:04.625 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:23:04.625 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:25:07.892 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:25:07.894 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:27:13.072 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:27:13.073 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:29:16.169 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:29:16.169 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:31:21.351 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:31:21.353 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:33:26.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:33:26.542 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:35:31.725 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:35:31.725 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:35:31.798 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-14T19:35:42.051 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-14T19:35:42.052 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T19:35:42.102 DEBUG:teuthology.nuke:shortname: smithi098 2023-07-14T19:35:42.103 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:35:42.122 DEBUG:teuthology.nuke:shortname: smithi101 2023-07-14T19:35:42.122 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:35:42.127 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338055', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:51.079221', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-14T19:35:42.146 DEBUG:teuthology.nuke:shortname: smithi139 2023-07-14T19:35:42.147 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:35:42.150 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338055', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:51.082748', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T19:35:42.173 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338055', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:51:51.081325', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T19:35:42.179 INFO:teuthology.orchestra.console:Power off smithi098 2023-07-14T19:35:42.180 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:35:42.200 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:35:42.218 INFO:teuthology.orchestra.console:Power off smithi101 2023-07-14T19:35:42.219 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:35:42.239 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:35:42.260 INFO:teuthology.orchestra.console:Power off smithi139 2023-07-14T19:35:42.261 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:35:42.332 INFO:teuthology.orchestra.console:Power off for smithi098 completed 2023-07-14T19:35:47.442 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:35:52.458 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:35:52.498 INFO:teuthology.lock.ops:unlocked: smithi098.front.sepia.ceph.com 2023-07-14T19:35:56.468 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:35:57.568 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:35:57.780 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2023-07-14T19:35:58.953 INFO:teuthology.lock.ops:unlocked: smithi101.front.sepia.ceph.com 2023-07-14T19:36:01.580 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:36:06.692 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:36:11.808 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:36:16.920 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:36:22.032 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:36:27.144 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:36:32.258 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:36:32.858 INFO:teuthology.orchestra.console:Power off for smithi139 completed 2023-07-14T19:36:43.015 INFO:teuthology.lock.ops:unlocked: smithi139.front.sepia.ceph.com