2023-07-15T19:26:22.205 INFO:root:teuthology version: 0.0.1.dev147+gff9991b 2023-07-15T19:26:22.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:26:22.207 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:26:22.413 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd21e5fe8b0>, signals=[15]) 2023-07-15T19:26:22.414 INFO:teuthology.lock.ops:Start node 'smithi045.front.sepia.ceph.com' reimaging 2023-07-15T19:26:22.415 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: reset os type and version on server 2023-07-15T19:26:22.415 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-07-15T19:26:22.440 INFO:teuthology.lock.ops:Start node 'smithi053.front.sepia.ceph.com' reimaging 2023-07-15T19:26:22.441 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: reset os type and version on server 2023-07-15T19:26:22.441 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-07-15T19:26:22.442 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /host {"name": "smithi045"} 2023-07-15T19:26:24.123 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /host {"name": "smithi053"} 2023-07-15T19:26:29.195 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-15T19:26:29.199 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-15T19:26:29.246 DEBUG:teuthology.provision.fog.smithi053:Requesting image smithi_centos_8.stream (ID 3) 2023-07-15T19:26:29.246 DEBUG:teuthology.provision.fog.smithi053:FOG request: PUT /host/158 {"imageID": 3} 2023-07-15T19:26:29.254 DEBUG:teuthology.provision.fog.smithi045:Requesting image smithi_centos_8.stream (ID 3) 2023-07-15T19:26:29.254 DEBUG:teuthology.provision.fog.smithi045:FOG request: PUT /host/150 {"imageID": 3} 2023-07-15T19:26:29.304 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of centos 8.stream 2023-07-15T19:26:29.305 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-07-15T19:26:29.307 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of centos 8.stream 2023-07-15T19:26:29.307 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:26:29.372 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /tasktype {"name": "deploy"} 2023-07-15T19:26:29.385 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /tasktype {"name": "deploy"} 2023-07-15T19:26:29.416 DEBUG:teuthology.provision.fog.smithi053:FOG request: POST /host/158/task {"taskTypeID": 1} 2023-07-15T19:26:29.441 DEBUG:teuthology.provision.fog.smithi045:FOG request: POST /host/150/task {"taskTypeID": 1} 2023-07-15T19:26:29.477 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-07-15T19:26:29.503 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:26:29.604 INFO:teuthology.orchestra.console:Power off smithi053 2023-07-15T19:26:29.605 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-15T19:26:29.624 INFO:teuthology.orchestra.console:Power off smithi045 2023-07-15T19:26:29.625 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-15T19:26:29.634 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:29.653 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:33.668 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:33.682 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:38.782 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:38.793 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:43.896 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:43.905 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:49.008 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:49.018 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:54.121 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:54.130 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:59.236 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:26:59.246 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:04.349 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:04.359 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:09.462 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:09.471 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:14.573 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:14.583 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:14.785 INFO:teuthology.orchestra.console:Power off for smithi053 completed 2023-07-15T19:27:14.886 INFO:teuthology.orchestra.console:Power on smithi053 2023-07-15T19:27:14.886 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-15T19:27:14.910 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:18.936 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:19.696 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:24.048 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:24.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:25.020 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2023-07-15T19:27:25.121 INFO:teuthology.orchestra.console:Power on smithi045 2023-07-15T19:27:25.121 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-15T19:27:25.144 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:29.160 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:29.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:34.272 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:34.281 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:39.384 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:39.393 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:44.497 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:44.507 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:44.720 INFO:teuthology.orchestra.console:Power on for smithi045 completed 2023-07-15T19:27:44.821 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish 2023-07-15T19:27:44.822 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:27:53.520 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:27:58.631 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:28:03.745 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:28:04.599 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:28:08.857 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T19:28:09.067 INFO:teuthology.orchestra.console:Power on for smithi053 completed 2023-07-15T19:28:09.168 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish 2023-07-15T19:28:09.169 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-07-15T19:28:19.696 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:28:29.702 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-07-15T19:28:49.809 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:28:49.810 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-07-15T19:29:04.920 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-07-15T19:29:04.922 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:29:30.020 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:29:30.021 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-07-15T19:29:45.100 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-07-15T19:29:49.992 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:29:50.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:29:53.168 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-07-15T19:29:59.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:30:05.084 DEBUG:teuthology.provision.fog.smithi045:FOG request: GET /task/active None 2023-07-15T19:30:10.093 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-07-15T19:30:10.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:30:16.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:30:21.101 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-07-15T19:30:24.176 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-07-15T19:30:27.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:30:30.160 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-07-15T19:30:30.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:30:36.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:30:44.247 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2023-07-15T19:30:48.560 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-07-15T19:30:50.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:30:55.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:30:58.320 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-07-15T19:31:04.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:31:06.552 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-15T19:31:06.553 DEBUG:teuthology.orchestra.run.smithi053:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-15T19:31:21.827 INFO:teuthology.provision.fog.smithi053:Node is ready 2023-07-15T19:31:21.827 DEBUG:teuthology.orchestra.run.smithi053:> hostname 2023-07-15T19:31:21.892 INFO:teuthology.orchestra.run.smithi053.stdout:smithi193 2023-07-15T19:31:21.894 DEBUG:teuthology.orchestra.run.smithi053:> grep smithi193 /etc/hosts 2023-07-15T19:31:21.954 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.193 smithi193.front.sepia.ceph.com smithi193 2023-07-15T19:31:21.954 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053 2023-07-15T19:31:22.099 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi193/smithi053/g' /etc/hosts 2023-07-15T19:31:22.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/172.21.15.193/172.21.15.53/g' /etc/hosts 2023-07-15T19:31:22.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi193/smithi053/g' /etc/hostname 2023-07-15T19:31:22.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053 2023-07-15T19:31:22.342 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-07-15T19:31:22.402 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2023-07-15T19:31:22.402 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8" 2023-07-15T19:31:22.403 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2023-07-15T19:31:22.403 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2023-07-15T19:31:22.403 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8" 2023-07-15T19:31:22.404 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2023-07-15T19:31:22.404 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-15T19:31:22.404 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-07-15T19:31:22.404 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-15T19:31:22.405 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2023-07-15T19:31:22.405 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-15T19:31:22.405 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-15T19:31:22.406 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-15T19:31:22.407 INFO:teuthology.provision.fog.smithi053:Deploy complete! 2023-07-15T19:31:22.408 DEBUG:teuthology.parallel:result is None 2023-07-15T19:31:55.288 WARNING:teuthology.provision.fog.smithi045:timed out 2023-07-15T19:32:01.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:32:01.559 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-15T19:32:01.560 DEBUG:teuthology.orchestra.run.smithi045:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-15T19:32:01.810 INFO:teuthology.provision.fog.smithi045:Node is ready 2023-07-15T19:32:01.811 DEBUG:teuthology.orchestra.run.smithi045:> hostname 2023-07-15T19:32:01.872 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com 2023-07-15T19:32:01.873 DEBUG:teuthology.orchestra.run.smithi045:> grep smithi045.front.sepia.ceph.com /etc/hosts 2023-07-15T19:32:01.932 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045 2023-07-15T19:32:01.933 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname smithi045 2023-07-15T19:32:02.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/smithi045.front.sepia.ceph.com/smithi045/g' /etc/hosts 2023-07-15T19:32:02.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/172.21.15.45/172.21.15.45/g' /etc/hosts 2023-07-15T19:32:02.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/smithi045.front.sepia.ceph.com/smithi045/g' /etc/hostname 2023-07-15T19:32:02.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname smithi045 2023-07-15T19:32:02.285 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2023-07-15T19:32:02.345 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2023-07-15T19:32:02.347 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8" 2023-07-15T19:32:02.348 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2023-07-15T19:32:02.348 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2023-07-15T19:32:02.348 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8" 2023-07-15T19:32:02.348 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2023-07-15T19:32:02.348 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-15T19:32:02.348 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-07-15T19:32:02.349 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-15T19:32:02.349 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2023-07-15T19:32:02.349 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-15T19:32:02.349 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-15T19:32:02.349 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-15T19:32:02.349 INFO:teuthology.provision.fog.smithi045:Deploy complete! 2023-07-15T19:32:02.350 DEBUG:teuthology.parallel:result is None 2023-07-15T19:32:07.526 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com 2023-07-15T19:32:07.527 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-15T19:32:12.562 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com 2023-07-15T19:32:12.645 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-15T19:32:17.671 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server 2023-07-15T19:32:17.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-07-15T19:32:17.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:32:22.747 DEBUG:teuthology.orchestra.run.smithi045:> true 2023-07-15T19:32:22.840 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2023-07-15T19:32:22.841 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2023-07-15T19:32:22.899 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2023-07-15T19:32:22.900 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2023-07-15T19:32:22.959 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2023-07-15T19:32:22.959 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8" 2023-07-15T19:32:22.959 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2023-07-15T19:32:22.959 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2023-07-15T19:32:22.959 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8" 2023-07-15T19:32:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2023-07-15T19:32:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-15T19:32:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2023-07-15T19:32:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-15T19:32:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2023-07-15T19:32:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-15T19:32:22.960 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-15T19:32:22.961 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-15T19:32:22.961 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2023-07-15T19:32:27.994 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server 2023-07-15T19:32:27.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-07-15T19:32:27.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-15T19:32:28.065 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-07-15T19:32:28.209 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-07-15T19:32:28.210 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2023-07-15T19:32:28.269 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2023-07-15T19:32:28.270 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-07-15T19:32:28.332 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2023-07-15T19:32:28.332 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8" 2023-07-15T19:32:28.332 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2023-07-15T19:32:28.332 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2023-07-15T19:32:28.333 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8" 2023-07-15T19:32:28.333 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2023-07-15T19:32:28.333 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-15T19:32:28.333 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-07-15T19:32:28.333 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-15T19:32:28.333 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2023-07-15T19:32:28.333 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-15T19:32:28.334 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-15T19:32:28.334 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-15T19:32:28.334 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-07-15T19:32:33.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:32:33.359 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:32:38.481 INFO:teuthology.dispatcher.supervisor:Running job 7339364 2023-07-15T19:32:38.482 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_ff9991b7ed74eda9de4a5215d6d7d2a72f7b8d2b/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-14_23:47:33-orch-reef-distro-default-smithi/7339364 --name yuriw-2023-07-14_23:47:33-orch-reef-distro-default-smithi --description orch/cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} -- /home/teuthworker/archive/yuriw-2023-07-14_23:47:33-orch-reef-distro-default-smithi/7339364/orig.config.yaml 2023-07-15T19:32:38.487 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-14_23:47:33-orch-reef-distro-default-smithi/7339364 2023-07-15T19:32:38.488 INFO:teuthology.dispatcher.supervisor:Job PID: 13727 2023-07-15T19:32:38.488 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-15T19:34:38.588 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:34:38.590 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:36:43.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:36:43.773 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:38:48.953 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:38:48.953 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:40:54.127 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:40:54.128 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:42:54.313 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:42:54.314 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:44:59.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:44:59.490 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:47:04.665 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:47:04.666 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:49:09.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:49:09.853 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:51:15.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:51:15.037 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:53:20.209 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:53:20.210 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:55:25.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:55:25.385 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:57:30.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:57:30.562 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T19:59:32.561 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T19:59:32.562 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:01:37.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:01:37.737 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:03:42.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:03:42.916 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:05:47.717 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:05:47.718 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:07:52.888 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:07:52.889 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:09:58.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:09:58.070 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:12:00.801 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:12:00.802 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:14:04.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:14:04.386 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:16:07.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:16:07.606 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:18:12.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:18:12.786 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:20:17.965 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:20:17.966 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:22:20.950 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:22:20.951 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:24:26.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-15T20:24:26.133 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-15T20:24:31.217 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-15T20:24:40.697 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-15T20:24:40.697 INFO:teuthology.nuke:Checking targets against current locks 2023-07-15T20:24:44.300 DEBUG:teuthology.nuke:shortname: smithi045 2023-07-15T20:24:44.301 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-15T20:24:44.322 DEBUG:teuthology.nuke:shortname: smithi053 2023-07-15T20:24:44.323 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-15T20:24:44.326 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-14_23:47:33-orch-reef-distro-default-smithi/7339364', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-15 19:26:16.291802', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGxcH/Wb4pRBCYRgJ3GLsWrHj/MxccHlsg0oJH4MbxL6/cEjYNmSPALZl+M6/wCbjEXeJRKyp/5XE2hZCoV5hAZ2/MqK+WDc9NmQ97OGD9kjN/bEO9wEQiC8YEFgcL0eb3NpoVZKJuWxpkE6w3YsBvsCWduHG/l6PBhB0QZeupUCqngy59kPlngbk4g/XpVcihy+d0X/BH4PEVlD5LZQuKRZZajSKyUp0juUr0fO8RAq4XlmGQkYu8lYJrxAOm/8xtOnEQuuYlK7t5cyPTIbCF6/5FKWlUbqE0jwxc+TRCglW+YDpgh/CI3Nm5yMigDE3Ml9cJPbV4XRJpOKL+UVwlzdcH14GwemkBaF9scvE/ZG9EtnnotEp+mYuRLI+EewibBiD9Bu3vRO0bmmiHYIHUiktflNtUt1v+r3wJ3jb/uuSpbyjmUj2rGxnON1puARjDkeiRvoLAdWuGuEX37bYnZnKOxqN0XObV0SVodCLRPH7qqIB2vqHrjWrsOFB7I48='} 2023-07-15T20:24:44.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-14_23:47:33-orch-reef-distro-default-smithi/7339364', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-15 19:26:16.290311', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzZZZkbH0Biv+uY4p23yYHC6t+zhNxe/ydFhkKprzlhXjrVqX6vLAeaI+g/rSl9VVAYfW7OUe8ngKE08X//XBooksqR2NE3H5zhitoVc3NLYVVVZIsjYriWFHwb/n61fuk6WHkBAtKYA60HJE06p2KibL3O7g2dxRlEH9fmEWR91Nx2yR1ap7lktDTm5djlV11DTjNDy82ZAkS2kMNN0s7+7B89CQAWbp2SmL3/65iSayOfjxmMN9OYgtgC5AfNXqwLJE6WxdMQWAiZBJ3VPD3tPEY+ZftvOk3Nnn9LgGpyVEeZqsmdlSht8l12kdoxfqk+Iy7iSD0bd/K3fAwCZhw5h/IYIydsIQiuJGmjm1DRCjuOsbpdzuw7zDpWmFJB5Yz2d+m7GnhFp5ycGgppUOPTBrnWgOFoSXjKz4GpU/Idqx9DxgsklUemd0NYwhzxR8aYuSZThpt483GWDeMJFYhEZ4nGf0RNMU6TNTOQZB+jrLwdJLRHczd2aPxBOn2V88='} 2023-07-15T20:24:44.376 INFO:teuthology.orchestra.console:Power off smithi045 2023-07-15T20:24:44.377 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-15T20:24:44.401 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T20:24:44.416 INFO:teuthology.orchestra.console:Power off smithi053 2023-07-15T20:24:44.417 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-15T20:24:44.440 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-15T20:24:44.529 INFO:teuthology.orchestra.console:Power off for smithi045 completed 2023-07-15T20:24:44.565 INFO:teuthology.orchestra.console:Power off for smithi053 completed 2023-07-15T20:24:44.691 INFO:teuthology.lock.ops:unlocked: smithi045.front.sepia.ceph.com 2023-07-15T20:24:44.725 INFO:teuthology.lock.ops:unlocked: smithi053.front.sepia.ceph.com