2023-07-14T17:53:13.990 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T17:53:13.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T17:53:13.992 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T17:53:14.208 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f42601ce8b0>, signals=[15]) 2023-07-14T17:53:14.210 INFO:teuthology.lock.ops:Start node 'smithi137.front.sepia.ceph.com' reimaging 2023-07-14T17:53:14.210 INFO:teuthology.lock.ops:Updating [smithi137.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T17:53:14.211 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2023-07-14T17:53:14.237 INFO:teuthology.lock.ops:Start node 'smithi145.front.sepia.ceph.com' reimaging 2023-07-14T17:53:14.237 INFO:teuthology.lock.ops:Updating [smithi145.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T17:53:14.237 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2023-07-14T17:53:14.239 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /host {"name": "smithi137"} 2023-07-14T17:53:14.262 INFO:teuthology.lock.ops:Start node 'smithi180.front.sepia.ceph.com' reimaging 2023-07-14T17:53:14.262 INFO:teuthology.lock.ops:Updating [smithi180.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T17:53:14.263 INFO:teuthology.lock.ops:Updating smithi180.front.sepia.ceph.com on lock server 2023-07-14T17:53:14.265 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /host {"name": "smithi145"} 2023-07-14T17:53:14.287 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /host {"name": "smithi180"} 2023-07-14T17:53:14.310 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-14T17:53:14.346 DEBUG:teuthology.provision.fog.smithi137:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-14T17:53:14.347 DEBUG:teuthology.provision.fog.smithi137:FOG request: PUT /host/242 {"imageID": 5} 2023-07-14T17:53:14.352 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-14T17:53:14.354 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-07-14T17:53:14.399 DEBUG:teuthology.provision.fog.smithi145:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-14T17:53:14.400 DEBUG:teuthology.provision.fog.smithi145:FOG request: PUT /host/250 {"imageID": 5} 2023-07-14T17:53:14.404 DEBUG:teuthology.provision.fog.smithi180:Requesting image smithi_rhel_8.6 (ID 5) 2023-07-14T17:53:14.404 DEBUG:teuthology.provision.fog.smithi180:FOG request: PUT /host/285 {"imageID": 5} 2023-07-14T17:53:14.411 INFO:teuthology.provision.fog.smithi137:Scheduling deploy of rhel 8.6 2023-07-14T17:53:14.411 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:53:14.446 INFO:teuthology.provision.fog.smithi180:Scheduling deploy of rhel 8.6 2023-07-14T17:53:14.446 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:53:14.449 INFO:teuthology.provision.fog.smithi145:Scheduling deploy of rhel 8.6 2023-07-14T17:53:14.449 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:53:14.482 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T17:53:14.506 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T17:53:14.509 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T17:53:14.518 DEBUG:teuthology.provision.fog.smithi137:FOG request: POST /host/242/task {"taskTypeID": 1} 2023-07-14T17:53:14.543 DEBUG:teuthology.provision.fog.smithi180:FOG request: POST /host/285/task {"taskTypeID": 1} 2023-07-14T17:53:14.546 DEBUG:teuthology.provision.fog.smithi145:FOG request: POST /host/250/task {"taskTypeID": 1} 2023-07-14T17:53:14.576 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:53:14.596 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:53:14.599 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:53:14.681 INFO:teuthology.orchestra.console:Power off smithi137 2023-07-14T17:53:14.682 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T17:53:14.710 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:14.738 INFO:teuthology.orchestra.console:Power off smithi180 2023-07-14T17:53:14.739 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T17:53:14.757 INFO:teuthology.orchestra.console:Power off smithi145 2023-07-14T17:53:14.758 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T17:53:14.769 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:14.781 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:14.868 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2023-07-14T17:53:14.905 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2023-07-14T17:53:14.970 INFO:teuthology.orchestra.console:Power on smithi137 2023-07-14T17:53:14.971 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T17:53:14.994 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:15.007 INFO:teuthology.orchestra.console:Power on smithi145 2023-07-14T17:53:15.008 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T17:53:18.800 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:20.006 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:20.033 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:22.912 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:25.044 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:25.116 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:28.023 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:30.156 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:30.229 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:33.136 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:35.268 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:35.339 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:35.479 INFO:teuthology.orchestra.console:Power on for smithi145 completed 2023-07-14T17:53:35.580 INFO:teuthology.provision.fog.smithi145:Waiting for deploy to finish 2023-07-14T17:53:35.581 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:53:40.589 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:40.698 INFO:teuthology.orchestra.console:Power on for smithi137 completed 2023-07-14T17:53:40.799 INFO:teuthology.provision.fog.smithi137:Waiting for deploy to finish 2023-07-14T17:53:40.800 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:53:49.813 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:54.925 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:53:55.672 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:54:00.040 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:54:00.876 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:54:15.885 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:54:20.994 INFO:teuthology.orchestra.console:Power off for smithi180 completed 2023-07-14T17:54:21.095 INFO:teuthology.orchestra.console:Power on smithi180 2023-07-14T17:54:21.095 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T17:54:26.123 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:54:31.136 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:54:35.968 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:54:35.971 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:54:40.499 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:54:45.613 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:54:50.725 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T17:54:51.607 INFO:teuthology.orchestra.console:Power on for smithi180 completed 2023-07-14T17:54:51.708 INFO:teuthology.provision.fog.smithi180:Waiting for deploy to finish 2023-07-14T17:54:51.709 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:54:56.717 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:54:56.719 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:55:21.812 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:55:21.814 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:55:21.815 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:55:46.924 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:55:46.925 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:55:46.926 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:56:07.012 DEBUG:teuthology.provision.fog.smithi145:FOG request: GET /task/active None 2023-07-14T17:56:07.014 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:56:17.025 DEBUG:teuthology.provision.fog.smithi137:FOG request: GET /task/active None 2023-07-14T17:56:21.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:56:21.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:56:24.144 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-14T17:56:24.208 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-07-14T17:56:30.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:56:30.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:56:33.236 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-14T17:56:33.264 WARNING:teuthology.provision.fog.smithi137:[Errno None] Unable to connect to port 22 on 172.21.15.137 2023-07-14T17:56:36.140 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:56:39.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:56:44.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:56:49.249 WARNING:teuthology.provision.fog.smithi145:[Errno None] Unable to connect to port 22 on 172.21.15.145 2023-07-14T17:56:51.202 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:56:56.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:57:11.280 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:57:31.352 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:57:49.260 WARNING:teuthology.provision.fog.smithi137:timed out 2023-07-14T17:57:51.320 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:57:55.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:57:55.622 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T17:57:55.623 DEBUG:teuthology.orchestra.run.smithi137:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T17:57:55.894 INFO:teuthology.provision.fog.smithi137:Node is ready 2023-07-14T17:57:55.894 DEBUG:teuthology.orchestra.run.smithi137:> hostname 2023-07-14T17:57:55.911 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137.front.sepia.ceph.com 2023-07-14T17:57:55.912 DEBUG:teuthology.orchestra.run.smithi137:> grep smithi137.front.sepia.ceph.com /etc/hosts 2023-07-14T17:57:55.969 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137 smithi137.front.sepia.ceph.com smithi137 2023-07-14T17:57:55.970 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137 2023-07-14T17:57:56.091 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hosts 2023-07-14T17:57:56.119 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/172.21.15.137/172.21.15.137/g' /etc/hosts 2023-07-14T17:57:56.186 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i -e 's/smithi137.front.sepia.ceph.com/smithi137/g' /etc/hostname 2023-07-14T17:57:56.212 WARNING:teuthology.provision.fog.smithi145:timed out 2023-07-14T17:57:56.253 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname smithi137 2023-07-14T17:57:56.319 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2023-07-14T17:57:56.375 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T17:57:56.375 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T17:57:56.376 INFO:teuthology.orchestra.run.smithi137.stdout:ID="rhel" 2023-07-14T17:57:56.376 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="fedora" 2023-07-14T17:57:56.376 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="8.6" 2023-07-14T17:57:56.376 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el8" 2023-07-14T17:57:56.376 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T17:57:56.376 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2023-07-14T17:57:56.376 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T17:57:56.376 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T17:57:56.377 INFO:teuthology.orchestra.run.smithi137.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T17:57:56.377 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T17:57:56.377 INFO:teuthology.orchestra.run.smithi137.stdout: 2023-07-14T17:57:56.377 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T17:57:56.377 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T17:57:56.377 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T17:57:56.377 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T17:57:56.378 INFO:teuthology.provision.fog.smithi137:Deploy complete! 2023-07-14T17:57:56.379 DEBUG:teuthology.parallel:result is None 2023-07-14T17:58:02.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T17:58:02.675 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T17:58:02.675 DEBUG:teuthology.orchestra.run.smithi145:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T17:58:02.982 INFO:teuthology.provision.fog.smithi145:Node is ready 2023-07-14T17:58:02.983 DEBUG:teuthology.orchestra.run.smithi145:> hostname 2023-07-14T17:58:03.001 INFO:teuthology.orchestra.run.smithi145.stdout:smithi145.front.sepia.ceph.com 2023-07-14T17:58:03.001 DEBUG:teuthology.orchestra.run.smithi145:> grep smithi145.front.sepia.ceph.com /etc/hosts 2023-07-14T17:58:03.060 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.15.145 smithi145.front.sepia.ceph.com smithi145 2023-07-14T17:58:03.060 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname smithi145 2023-07-14T17:58:03.162 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/smithi145.front.sepia.ceph.com/smithi145/g' /etc/hosts 2023-07-14T17:58:03.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/172.21.15.145/172.21.15.145/g' /etc/hosts 2023-07-14T17:58:03.257 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i -e 's/smithi145.front.sepia.ceph.com/smithi145/g' /etc/hostname 2023-07-14T17:58:03.324 DEBUG:teuthology.orchestra.run.smithi145:> sudo hostname smithi145 2023-07-14T17:58:03.390 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2023-07-14T17:58:03.447 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Red Hat Enterprise Linux" 2023-07-14T17:58:03.447 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8.6 (Ootpa)" 2023-07-14T17:58:03.447 INFO:teuthology.orchestra.run.smithi145.stdout:ID="rhel" 2023-07-14T17:58:03.448 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="fedora" 2023-07-14T17:58:03.448 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8.6" 2023-07-14T17:58:03.448 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2023-07-14T17:58:03.448 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-14T17:58:03.448 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2023-07-14T17:58:03.448 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-14T17:58:03.449 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.redhat.com/" 2023-07-14T17:58:03.449 INFO:teuthology.orchestra.run.smithi145.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-14T17:58:03.449 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T17:58:03.449 INFO:teuthology.orchestra.run.smithi145.stdout: 2023-07-14T17:58:03.449 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-14T17:58:03.449 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-14T17:58:03.449 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-14T17:58:03.449 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-14T17:58:03.450 INFO:teuthology.provision.fog.smithi145:Deploy complete! 2023-07-14T17:58:03.450 DEBUG:teuthology.parallel:result is None 2023-07-14T17:58:06.408 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:58:26.496 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:58:42.852 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:58:57.944 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:59:18.036 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:59:33.112 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T17:59:53.028 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:00:13.111 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:00:28.824 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:00:48.896 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:01:08.964 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:01:24.036 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:01:39.112 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:01:59.188 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:02:17.584 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:02:32.672 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:02:52.756 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:03:07.825 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:03:22.924 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:03:43.005 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:04:03.080 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:04:18.140 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:04:38.224 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:04:58.300 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:05:13.380 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:05:28.479 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:05:43.574 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:05:58.652 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:06:13.725 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:06:32.988 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:06:53.104 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:07:13.220 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:07:28.324 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:07:43.428 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:08:03.516 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:08:18.660 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:08:38.736 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:08:58.800 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:09:18.868 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:09:38.952 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:09:59.016 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:10:14.084 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:10:31.428 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:10:46.511 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:11:06.604 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:11:21.700 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:11:41.785 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:12:01.848 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:12:21.928 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:12:42.016 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:12:57.100 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:13:14.420 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:13:34.512 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:13:54.573 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:14:14.676 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:14:29.776 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:14:49.838 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:15:09.932 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:15:30.008 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:15:48.092 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:16:03.184 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:16:23.280 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:16:38.364 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:16:55.152 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:17:10.247 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:17:30.358 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:17:45.460 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:18:00.556 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:18:15.640 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:18:35.628 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:18:55.736 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:19:15.820 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:19:35.917 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:19:55.944 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:20:16.044 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:20:31.136 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:20:46.222 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:21:06.290 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:21:26.336 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:21:46.400 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:22:06.465 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:22:26.536 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:22:43.571 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:23:03.669 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:23:23.752 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:23:43.835 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:24:03.908 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:24:18.972 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:24:36.928 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:24:57.013 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:25:17.089 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:25:37.188 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:25:57.280 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:26:17.368 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:26:32.450 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:26:52.527 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:27:07.604 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:27:27.684 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:27:47.752 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:28:07.084 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:28:27.176 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:28:47.264 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:29:07.353 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:29:27.448 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:29:47.532 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:30:07.632 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:30:27.736 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:30:42.844 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:31:02.932 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:31:18.034 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:31:38.132 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:31:58.207 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:32:16.280 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:32:31.408 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:32:48.640 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:33:08.749 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:33:28.860 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:33:48.962 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:34:09.092 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:34:29.168 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:34:49.240 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:35:04.304 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:35:19.380 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:35:39.449 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:35:54.504 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:36:09.576 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:36:29.672 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:36:44.748 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:36:59.812 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:37:19.880 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:37:39.969 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:38:00.046 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:38:20.132 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:38:35.216 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:38:55.290 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:39:15.365 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:39:35.436 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:39:55.516 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:40:15.600 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:40:30.676 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:40:50.765 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:41:05.840 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:41:25.917 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:41:40.992 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:41:56.052 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:42:16.125 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:42:31.204 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:42:46.281 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:43:06.368 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:43:26.432 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:43:46.500 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:44:01.568 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:44:21.652 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:44:41.728 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:45:01.796 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:45:21.860 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:45:41.951 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:45:57.028 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:46:17.096 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:46:37.188 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:46:57.312 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:47:17.420 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:47:32.516 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:47:52.640 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:48:12.744 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:48:29.084 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:48:49.192 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:49:09.296 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:49:24.441 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:49:44.524 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:49:59.588 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:50:19.660 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:50:35.612 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:50:55.688 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:51:10.784 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:51:25.884 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:51:45.968 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:52:04.036 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:52:24.134 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:52:44.236 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:52:59.788 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:53:14.905 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:53:34.900 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:53:54.998 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:54:15.092 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:54:30.192 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:54:50.288 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:55:10.376 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:55:30.448 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:55:49.071 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:56:04.153 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:56:24.248 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:56:44.336 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:56:59.424 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:57:14.524 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:57:34.608 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:57:54.684 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:58:14.344 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:58:34.432 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:58:54.060 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:59:14.146 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:59:29.337 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T18:59:49.332 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:00:09.408 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:00:29.504 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:00:44.576 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:00:59.644 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:01:14.716 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:01:29.776 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:01:49.784 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:02:09.849 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:02:29.916 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:02:50.000 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:03:10.076 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:03:25.155 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:03:45.236 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:04:05.336 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:04:25.424 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:04:40.512 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:05:00.605 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:05:16.028 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:05:31.136 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:05:46.292 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:06:06.372 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:06:26.452 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:06:46.544 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:07:06.648 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:07:21.744 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:07:38.136 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:07:53.269 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:08:13.369 DEBUG:teuthology.provision.fog.smithi180:FOG request: GET /task/active None 2023-07-14T19:08:14.488 DEBUG:teuthology.provision.fog.smithi180:Canceling deploy task with ID 384774 2023-07-14T19:08:14.489 DEBUG:teuthology.provision.fog.smithi180:FOG request: DELETE /task/cancel {"id": 384774} 2023-07-14T19:08:14.534 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/provision/fog.py", line 86, in create self.wait_for_deploy_task(task_id) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/provision/fog.py", line 259, in wait_for_deploy_task while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds 2023-07-14T19:08:14.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T19:08:14.535 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T19:08:16.472 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T19:08:21.465 DEBUG:teuthology.nuke:shortname: smithi137 2023-07-14T19:08:21.466 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:08:21.490 DEBUG:teuthology.nuke:shortname: smithi145 2023-07-14T19:08:21.491 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:08:21.494 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338011', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:53:13.152504', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T19:08:21.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338011', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:53:13.151354', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-14T19:08:21.519 DEBUG:teuthology.nuke:shortname: smithi180 2023-07-14T19:08:21.519 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T19:08:21.540 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi180.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338011', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 17:53:13.150124', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClSicqGWHaveeEZnp0mgVeorreKCb08RuxmwX4YnKbLuI1W1APFQGrD2+YqYh/s4rJpzs+Seh2T1l9KgNkUaetLag/+zFsbsmVTwg6kKiiVQmZB9Zn4jVOmApmIGPmf7v6Plxd1o4r0FjWTeeS3dBWUds/rMclPGPtwb3UqVpczO5W712aV6t1BE47GYhAwhlq6t9DpVYWu1ddb0Yw30PQCJINx8nBTwhao8RnRBQv0cx+mOmWFfysK8xzlzEL2Hfx+P6tugt7NHAJyy/qHQkuMoofgPkjQje8chSmfltBERtpGlGlGuHiQJ/mTRmQl5XWdEeJNGHPn5Lx1QzxUVrKhsGB4CGoKHt6Mtsqurqo/5Rd5mHvTuOogj/oDdGWp1h/7ev7L0UAGqM0hymneGG9WHaQbEFeSx+g1EOJs+9c5R+SF5Pd3n7t+nAoIkqrVxSDliJRmjGRTk3YCBVebhdDVJt4rGh+818DwDXEjNNdz5G2ypX2D/ys/TJqQUuszP8='} 2023-07-14T19:08:21.549 INFO:teuthology.orchestra.console:Power off smithi137 2023-07-14T19:08:21.550 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:08:21.571 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:21.588 INFO:teuthology.orchestra.console:Power off smithi145 2023-07-14T19:08:21.589 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:08:21.610 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:21.634 INFO:teuthology.orchestra.console:Power off smithi180 2023-07-14T19:08:21.635 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T19:08:21.659 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:25.604 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:25.647 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:25.684 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:30.718 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:30.758 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:30.797 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:30.932 INFO:teuthology.orchestra.console:Power off for smithi137 completed 2023-07-14T19:08:31.101 INFO:teuthology.lock.ops:unlocked: smithi137.front.sepia.ceph.com 2023-07-14T19:08:35.872 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:35.909 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi180.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T19:08:36.121 INFO:teuthology.orchestra.console:Power off for smithi180 completed 2023-07-14T19:08:46.260 INFO:teuthology.orchestra.console:Power off for smithi145 completed 2023-07-14T19:08:46.290 INFO:teuthology.lock.ops:unlocked: smithi180.front.sepia.ceph.com 2023-07-14T19:08:56.500 INFO:teuthology.lock.ops:unlocked: smithi145.front.sepia.ceph.com 2023-07-14T19:09:03.484 ERROR:teuthology.dispatcher.supervisor:Reimage failed 10 times ... marking machine down 2023-07-14T19:09:03.485 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/provision/fog.py", line 86, in create self.wait_for_deploy_task(task_id) File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/provision/fog.py", line 259, in wait_for_deploy_task while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (241) after waiting for 3600 seconds