2023-07-14T16:35:24.294 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e 2023-07-14T16:35:24.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:35:24.296 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:35:43.851 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f038bcb98b0>, signals=[15]) 2023-07-14T16:35:43.853 INFO:teuthology.lock.ops:Start node 'smithi172.front.sepia.ceph.com' reimaging 2023-07-14T16:35:43.853 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T16:35:43.853 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-07-14T16:35:43.871 INFO:teuthology.lock.ops:Start node 'smithi184.front.sepia.ceph.com' reimaging 2023-07-14T16:35:43.871 INFO:teuthology.lock.ops:Updating [smithi184.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T16:35:43.871 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2023-07-14T16:35:43.873 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /host {"name": "smithi172"} 2023-07-14T16:35:44.204 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /host {"name": "smithi184"} 2023-07-14T16:35:44.279 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T16:35:44.329 DEBUG:teuthology.provision.fog.smithi172:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T16:35:44.330 DEBUG:teuthology.provision.fog.smithi172:FOG request: PUT /host/277 {"imageID": 10} 2023-07-14T16:35:44.336 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T16:35:44.376 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of centos 9.stream 2023-07-14T16:35:44.376 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:35:44.381 DEBUG:teuthology.provision.fog.smithi184:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T16:35:44.382 DEBUG:teuthology.provision.fog.smithi184:FOG request: PUT /host/289 {"imageID": 10} 2023-07-14T16:35:44.426 INFO:teuthology.provision.fog.smithi184:Scheduling deploy of centos 9.stream 2023-07-14T16:35:44.427 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:35:44.455 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T16:35:44.484 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T16:35:44.497 DEBUG:teuthology.provision.fog.smithi172:FOG request: POST /host/277/task {"taskTypeID": 1} 2023-07-14T16:35:44.516 DEBUG:teuthology.provision.fog.smithi184:FOG request: POST /host/289/task {"taskTypeID": 1} 2023-07-14T16:35:44.564 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:35:44.574 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:35:44.675 INFO:teuthology.orchestra.console:Power off smithi172 2023-07-14T16:35:44.676 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T16:35:44.693 INFO:teuthology.orchestra.console:Power off smithi184 2023-07-14T16:35:44.694 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T16:35:44.704 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:35:44.717 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:35:48.732 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:35:48.741 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:35:53.843 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:35:53.854 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:35:58.957 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:35:58.968 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:35:59.179 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2023-07-14T16:35:59.280 INFO:teuthology.orchestra.console:Power on smithi184 2023-07-14T16:35:59.281 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T16:36:04.073 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:04.308 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:09.188 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:09.321 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:14.300 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:14.431 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:19.413 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:19.543 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:19.624 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2023-07-14T16:36:19.725 INFO:teuthology.orchestra.console:Power on smithi172 2023-07-14T16:36:19.726 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T16:36:24.658 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:24.751 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:29.761 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:29.770 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:34.873 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:34.882 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:36:35.094 INFO:teuthology.orchestra.console:Power on for smithi184 completed 2023-07-14T16:36:35.195 INFO:teuthology.provision.fog.smithi184:Waiting for deploy to finish 2023-07-14T16:36:35.196 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:36:40.302 INFO:teuthology.orchestra.console:Power on for smithi172 completed 2023-07-14T16:36:40.402 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2023-07-14T16:36:40.403 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:36:55.280 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:37:00.490 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:37:10.378 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:37:19.776 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:37:30.456 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:37:39.868 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:37:50.452 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:37:54.936 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:38:09.840 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:38:19.849 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:38:39.920 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:38:39.922 DEBUG:teuthology.provision.fog.smithi184:FOG request: GET /task/active None 2023-07-14T16:38:39.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:38:48.048 WARNING:teuthology.provision.fog.smithi184:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-07-14T16:38:54.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:38:59.056 DEBUG:teuthology.provision.fog.smithi172:FOG request: GET /task/active None 2023-07-14T16:39:04.064 WARNING:teuthology.provision.fog.smithi184:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-07-14T16:39:04.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:39:10.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:39:12.176 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-07-14T16:39:13.136 WARNING:teuthology.provision.fog.smithi184:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-07-14T16:39:18.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:39:19.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:39:21.236 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-07-14T16:39:22.196 WARNING:teuthology.provision.fog.smithi184:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-07-14T16:39:27.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:39:32.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:39:37.249 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-07-14T16:39:43.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:39:51.312 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-07-14T16:39:57.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:40:05.392 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-07-14T16:40:11.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:40:34.935 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-07-14T16:40:37.252 WARNING:teuthology.provision.fog.smithi184:timed out 2023-07-14T16:40:40.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:40:43.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:40:43.517 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T16:40:43.518 DEBUG:teuthology.orchestra.run.smithi184:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T16:40:43.791 INFO:teuthology.provision.fog.smithi184:Node is ready 2023-07-14T16:40:43.791 DEBUG:teuthology.orchestra.run.smithi184:> hostname 2023-07-14T16:40:43.805 INFO:teuthology.orchestra.run.smithi184.stdout:smithi184.front.sepia.ceph.com 2023-07-14T16:40:43.806 DEBUG:teuthology.orchestra.run.smithi184:> grep smithi184.front.sepia.ceph.com /etc/hosts 2023-07-14T16:40:43.860 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.15.184 smithi184.front.sepia.ceph.com smithi184 2023-07-14T16:40:43.861 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname smithi184 2023-07-14T16:40:43.983 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i -e 's/smithi184.front.sepia.ceph.com/smithi184/g' /etc/hosts 2023-07-14T16:40:44.011 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i -e 's/172.21.15.184/172.21.15.184/g' /etc/hosts 2023-07-14T16:40:44.016 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2023-07-14T16:40:44.078 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i -e 's/smithi184.front.sepia.ceph.com/smithi184/g' /etc/hostname 2023-07-14T16:40:44.145 DEBUG:teuthology.orchestra.run.smithi184:> sudo hostname smithi184 2023-07-14T16:40:44.211 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2023-07-14T16:40:44.265 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2023-07-14T16:40:44.265 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2023-07-14T16:40:44.265 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2023-07-14T16:40:44.265 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2023-07-14T16:40:44.265 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2023-07-14T16:40:44.266 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2023-07-14T16:40:44.266 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T16:40:44.266 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2023-07-14T16:40:44.266 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2023-07-14T16:40:44.266 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T16:40:44.266 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2023-07-14T16:40:44.266 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T16:40:44.267 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T16:40:44.267 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T16:40:44.267 INFO:teuthology.provision.fog.smithi184:Deploy complete! 2023-07-14T16:40:44.269 DEBUG:teuthology.parallel:result is None 2023-07-14T16:40:50.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:40:55.275 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T16:40:55.276 DEBUG:teuthology.orchestra.run.smithi172:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T16:41:10.395 INFO:teuthology.provision.fog.smithi172:Node is ready 2023-07-14T16:41:10.395 DEBUG:teuthology.orchestra.run.smithi172:> hostname 2023-07-14T16:41:10.413 INFO:teuthology.orchestra.run.smithi172.stdout:smithi197 2023-07-14T16:41:10.413 DEBUG:teuthology.orchestra.run.smithi172:> grep smithi197 /etc/hosts 2023-07-14T16:41:10.470 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197 2023-07-14T16:41:10.471 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172 2023-07-14T16:41:10.599 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi197/smithi172/g' /etc/hosts 2023-07-14T16:41:10.630 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/172.21.15.197/172.21.15.172/g' /etc/hosts 2023-07-14T16:41:10.700 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi197/smithi172/g' /etc/hostname 2023-07-14T16:41:10.769 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172 2023-07-14T16:41:10.837 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-07-14T16:41:10.892 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2023-07-14T16:41:10.892 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2023-07-14T16:41:10.892 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2023-07-14T16:41:10.892 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2023-07-14T16:41:10.893 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2023-07-14T16:41:10.893 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2023-07-14T16:41:10.893 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T16:41:10.893 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-07-14T16:41:10.893 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2023-07-14T16:41:10.893 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T16:41:10.894 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2023-07-14T16:41:10.894 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T16:41:10.894 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T16:41:10.894 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T16:41:10.894 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2023-07-14T16:41:10.895 DEBUG:teuthology.parallel:result is None 2023-07-14T16:41:31.066 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2023-07-14T16:41:31.067 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-14T16:41:36.102 INFO:teuthology.lock.ops:Checking smithi184.front.sepia.ceph.com 2023-07-14T16:41:36.103 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-14T16:41:39.769 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2023-07-14T16:41:39.769 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-07-14T16:41:39.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:41:39.858 DEBUG:teuthology.orchestra.run.smithi172:> true 2023-07-14T16:41:40.027 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2023-07-14T16:41:40.027 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2023-07-14T16:41:40.082 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2023-07-14T16:41:40.082 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2023-07-14T16:41:40.136 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2023-07-14T16:41:40.137 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2023-07-14T16:41:40.137 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2023-07-14T16:41:40.137 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2023-07-14T16:41:40.137 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2023-07-14T16:41:40.137 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2023-07-14T16:41:40.138 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T16:41:40.138 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2023-07-14T16:41:40.138 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2023-07-14T16:41:40.138 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T16:41:40.138 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2023-07-14T16:41:40.138 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T16:41:40.138 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T16:41:40.139 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T16:41:40.139 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2023-07-14T16:41:40.167 INFO:teuthology.lock.ops:Updating [smithi184.front.sepia.ceph.com]: set os type and version on server 2023-07-14T16:41:40.167 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-07-14T16:41:40.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T16:41:40.251 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-07-14T16:41:40.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-07-14T16:41:40.423 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2023-07-14T16:41:40.477 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2023-07-14T16:41:40.477 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2023-07-14T16:41:40.532 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="CentOS Stream" 2023-07-14T16:41:40.532 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="9" 2023-07-14T16:41:40.533 INFO:teuthology.orchestra.run.smithi184.stdout:ID="centos" 2023-07-14T16:41:40.533 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="rhel fedora" 2023-07-14T16:41:40.533 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="9" 2023-07-14T16:41:40.533 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el9" 2023-07-14T16:41:40.533 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T16:41:40.533 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2023-07-14T16:41:40.533 INFO:teuthology.orchestra.run.smithi184.stdout:LOGO="fedora-logo-icon" 2023-07-14T16:41:40.534 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T16:41:40.534 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://centos.org/" 2023-07-14T16:41:40.534 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T16:41:40.534 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T16:41:40.534 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T16:41:40.535 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2023-07-14T16:41:40.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:41:40.559 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:41:40.658 INFO:teuthology.dispatcher.supervisor:Running job 7337710 2023-07-14T16:41:40.658 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7337710 --name yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi --description fs/upgrade/upgraded_client/{bluestore-bitmap centos_latest clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-from/pacific 1-mount/mount/fuse 2-workload/stress_tests/{0-client-upgrade 1-tests/blogbench}}} -- /home/teuthworker/archive/yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7337710/orig.config.yaml 2023-07-14T16:41:40.663 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7337710 2023-07-14T16:41:40.664 INFO:teuthology.dispatcher.supervisor:Job PID: 5428 2023-07-14T16:41:40.664 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-14T16:43:40.764 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:43:40.766 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:45:45.940 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:45:45.941 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:47:48.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:47:48.544 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:49:53.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:49:53.729 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:51:58.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:51:58.909 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:54:04.081 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:54:04.082 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:56:09.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T16:56:09.261 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T16:56:09.330 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-14T16:56:09.472 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-14T16:56:09.473 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T16:56:09.525 DEBUG:teuthology.nuke:shortname: smithi172 2023-07-14T16:56:09.526 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T16:56:09.544 DEBUG:teuthology.nuke:shortname: smithi184 2023-07-14T16:56:09.544 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T16:56:09.551 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7337710', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 16:35:23.497596', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP5Q5WlGCkiVChY0gFFULGmkAbX/4hx6xWgZacBZ8xwzoJ21ibzCLuNqU4k5fo6GFCJx4+9Jy7uDH5xZ/XnsSYI='} 2023-07-14T16:56:09.562 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-14_13:57:40-fs-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7337710', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 16:35:23.495913', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJQ7DwV9U4NQp1O/I216rdpSaqr5MwRSupvYnN+o0dg9jJNCEUulM0FQNV3KgJRasJpFAOFM3gWqsuTCxTab42w='} 2023-07-14T16:56:09.605 INFO:teuthology.orchestra.console:Power off smithi172 2023-07-14T16:56:09.606 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T16:56:09.622 INFO:teuthology.orchestra.console:Power off smithi184 2023-07-14T16:56:09.623 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T16:56:09.633 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:56:09.645 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T16:56:09.759 INFO:teuthology.orchestra.console:Power off for smithi172 completed 2023-07-14T16:56:09.767 INFO:teuthology.orchestra.console:Power off for smithi184 completed 2023-07-14T16:56:09.922 INFO:teuthology.lock.ops:unlocked: smithi172.front.sepia.ceph.com 2023-07-14T16:56:09.930 INFO:teuthology.lock.ops:unlocked: smithi184.front.sepia.ceph.com