2023-07-14T04:01:52.181 INFO:root:teuthology version: 0.0.1.dev143+g073e6eb 2023-07-14T04:01:52.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:01:52.183 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:01:56.574 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4a2d59b8b0>, signals=[15]) 2023-07-14T04:01:56.576 INFO:teuthology.lock.ops:Start node 'smithi122.front.sepia.ceph.com' reimaging 2023-07-14T04:01:56.576 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T04:01:56.577 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2023-07-14T04:01:56.606 INFO:teuthology.lock.ops:Start node 'smithi151.front.sepia.ceph.com' reimaging 2023-07-14T04:01:56.606 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: reset os type and version on server 2023-07-14T04:01:56.606 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2023-07-14T04:01:56.608 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /host {"name": "smithi122"} 2023-07-14T04:02:06.637 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /host {"name": "smithi151"} 2023-07-14T04:02:11.676 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T04:02:21.625 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-14T04:02:31.633 DEBUG:teuthology.provision.fog.smithi122:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T04:02:31.634 DEBUG:teuthology.provision.fog.smithi122:FOG request: PUT /host/227 {"imageID": 10} 2023-07-14T04:02:36.695 DEBUG:teuthology.provision.fog.smithi151:Requesting image smithi_centos_9.stream (ID 10) 2023-07-14T04:02:36.695 DEBUG:teuthology.provision.fog.smithi151:FOG request: PUT /host/256 {"imageID": 10} 2023-07-14T04:02:36.699 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of centos 9.stream 2023-07-14T04:02:36.699 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:02:36.756 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T04:02:41.765 INFO:teuthology.provision.fog.smithi151:Scheduling deploy of centos 9.stream 2023-07-14T04:02:41.766 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:02:46.823 DEBUG:teuthology.provision.fog.smithi122:FOG request: POST /host/227/task {"taskTypeID": 1} 2023-07-14T04:02:51.833 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /tasktype {"name": "deploy"} 2023-07-14T04:02:51.892 DEBUG:teuthology.provision.fog.smithi151:FOG request: POST /host/256/task {"taskTypeID": 1} 2023-07-14T04:02:51.902 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:02:51.944 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:02:51.998 INFO:teuthology.orchestra.console:Power off smithi122 2023-07-14T04:02:51.999 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T04:02:52.026 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:02:52.055 INFO:teuthology.orchestra.console:Power off smithi151 2023-07-14T04:02:52.056 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T04:02:52.081 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:02:56.068 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:02:56.110 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:01.189 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:01.235 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:01.448 INFO:teuthology.orchestra.console:Power off for smithi151 completed 2023-07-14T04:03:01.550 INFO:teuthology.orchestra.console:Power on smithi151 2023-07-14T04:03:01.552 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T04:03:01.591 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:05.636 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:06.303 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:10.753 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:11.418 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:11.632 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2023-07-14T04:03:11.734 INFO:teuthology.orchestra.console:Power on smithi122 2023-07-14T04:03:11.734 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-14T04:03:11.759 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:15.782 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:15.866 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:16.080 INFO:teuthology.orchestra.console:Power on for smithi151 completed 2023-07-14T04:03:16.181 INFO:teuthology.provision.fog.smithi151:Waiting for deploy to finish 2023-07-14T04:03:16.182 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:03:20.867 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T04:03:21.086 INFO:teuthology.orchestra.console:Power on for smithi122 completed 2023-07-14T04:03:21.194 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2023-07-14T04:03:21.195 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:03:35.949 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:03:41.258 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:03:56.012 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:04:01.322 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:04:16.076 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:04:21.378 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:04:36.140 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:04:41.298 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:04:56.204 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:05:01.372 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:05:11.276 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:05:16.444 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:05:26.344 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:05:31.504 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:05:46.423 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None 2023-07-14T04:05:49.751 DEBUG:teuthology.provision.fog.smithi122:FOG request: GET /task/active None 2023-07-14T04:05:54.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:05:59.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:02.900 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-07-14T04:06:02.901 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T04:06:08.907 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:08.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:11.984 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-07-14T04:06:11.985 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T04:06:17.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:18.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:21.072 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-07-14T04:06:21.073 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T04:06:27.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:27.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:30.160 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T04:06:30.161 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-07-14T04:06:36.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:41.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:46.174 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T04:06:49.233 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-07-14T04:06:52.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:06:57.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:07:15.568 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-07-14T04:07:20.688 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-07-14T04:07:21.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:07:26.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:07:31.698 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T04:07:31.698 DEBUG:teuthology.orchestra.run.smithi122:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T04:07:32.142 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-14T04:07:32.142 DEBUG:teuthology.orchestra.run.smithi151:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-14T04:07:46.862 INFO:teuthology.provision.fog.smithi122:Node is ready 2023-07-14T04:07:46.863 DEBUG:teuthology.orchestra.run.smithi122:> hostname 2023-07-14T04:07:46.881 INFO:teuthology.orchestra.run.smithi122.stdout:smithi197 2023-07-14T04:07:46.881 DEBUG:teuthology.orchestra.run.smithi122:> grep smithi197 /etc/hosts 2023-07-14T04:07:46.937 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197 2023-07-14T04:07:46.938 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122 2023-07-14T04:07:47.058 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi197/smithi122/g' /etc/hosts 2023-07-14T04:07:47.088 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/172.21.15.197/172.21.15.122/g' /etc/hosts 2023-07-14T04:07:47.156 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi197/smithi122/g' /etc/hostname 2023-07-14T04:07:47.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122 2023-07-14T04:07:47.294 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2023-07-14T04:07:47.352 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2023-07-14T04:07:47.353 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2023-07-14T04:07:47.353 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2023-07-14T04:07:47.353 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2023-07-14T04:07:47.353 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2023-07-14T04:07:47.353 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2023-07-14T04:07:47.353 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T04:07:47.354 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2023-07-14T04:07:47.354 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2023-07-14T04:07:47.354 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T04:07:47.354 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2023-07-14T04:07:47.354 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T04:07:47.354 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T04:07:47.354 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T04:07:47.355 INFO:teuthology.provision.fog.smithi122:Deploy complete! 2023-07-14T04:07:47.357 DEBUG:teuthology.parallel:result is None 2023-07-14T04:07:52.484 INFO:teuthology.provision.fog.smithi151:Node is ready 2023-07-14T04:07:52.484 DEBUG:teuthology.orchestra.run.smithi151:> hostname 2023-07-14T04:07:52.504 INFO:teuthology.orchestra.run.smithi151.stdout:smithi197 2023-07-14T04:07:52.504 DEBUG:teuthology.orchestra.run.smithi151:> grep smithi197 /etc/hosts 2023-07-14T04:07:52.562 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197 2023-07-14T04:07:52.562 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151 2023-07-14T04:07:52.691 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi197/smithi151/g' /etc/hosts 2023-07-14T04:07:52.724 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/172.21.15.197/172.21.15.151/g' /etc/hosts 2023-07-14T04:07:52.792 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi197/smithi151/g' /etc/hostname 2023-07-14T04:07:52.861 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151 2023-07-14T04:07:52.928 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2023-07-14T04:07:52.983 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2023-07-14T04:07:52.983 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2023-07-14T04:07:52.983 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2023-07-14T04:07:52.983 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2023-07-14T04:07:52.984 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2023-07-14T04:07:52.984 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2023-07-14T04:07:52.984 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T04:07:52.984 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2023-07-14T04:07:52.984 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2023-07-14T04:07:52.984 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T04:07:52.984 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2023-07-14T04:07:52.985 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T04:07:52.985 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T04:07:52.985 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T04:07:52.985 INFO:teuthology.provision.fog.smithi151:Deploy complete! 2023-07-14T04:07:52.986 DEBUG:teuthology.parallel:result is None 2023-07-14T04:08:03.149 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com 2023-07-14T04:08:03.150 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-14T04:08:08.178 INFO:teuthology.lock.ops:Checking smithi151.front.sepia.ceph.com 2023-07-14T04:08:08.179 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-14T04:08:13.212 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server 2023-07-14T04:08:13.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2023-07-14T04:08:13.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:08:18.304 DEBUG:teuthology.orchestra.run.smithi122:> true 2023-07-14T04:08:18.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2023-07-14T04:08:18.428 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2023-07-14T04:08:18.481 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2023-07-14T04:08:18.481 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2023-07-14T04:08:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2023-07-14T04:08:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2023-07-14T04:08:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2023-07-14T04:08:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2023-07-14T04:08:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2023-07-14T04:08:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2023-07-14T04:08:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T04:08:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2023-07-14T04:08:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2023-07-14T04:08:18.538 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T04:08:18.538 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2023-07-14T04:08:18.538 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T04:08:18.538 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T04:08:18.539 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T04:08:18.540 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2023-07-14T04:08:23.568 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: set os type and version on server 2023-07-14T04:08:23.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2023-07-14T04:08:23.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-14T04:08:28.697 DEBUG:teuthology.orchestra.run.smithi151:> true 2023-07-14T04:08:28.839 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2023-07-14T04:08:28.839 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2023-07-14T04:08:28.895 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2023-07-14T04:08:28.895 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2023-07-14T04:08:28.949 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2023-07-14T04:08:28.950 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2023-07-14T04:08:28.950 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2023-07-14T04:08:28.951 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2023-07-14T04:08:28.951 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2023-07-14T04:08:28.951 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2023-07-14T04:08:28.951 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-14T04:08:28.952 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2023-07-14T04:08:28.952 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2023-07-14T04:08:28.952 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-14T04:08:28.953 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2023-07-14T04:08:28.953 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-14T04:08:28.953 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-14T04:08:28.953 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-14T04:08:28.954 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2023-07-14T04:08:33.983 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:08:33.984 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:08:35.530 INFO:teuthology.dispatcher.supervisor:Running job 7336926 2023-07-14T04:08:35.530 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_073e6eb1231451146b4a01d0b85b4f490f71d266/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-13_23:12:00-rados-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336926 --name yuriw-2023-07-13_23:12:00-rados-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} -- /home/teuthworker/archive/yuriw-2023-07-13_23:12:00-rados-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336926/orig.config.yaml 2023-07-14T04:08:35.534 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-13_23:12:00-rados-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336926 2023-07-14T04:08:35.535 INFO:teuthology.dispatcher.supervisor:Job PID: 29813 2023-07-14T04:08:35.536 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-14T04:10:35.636 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:10:35.638 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:12:40.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:12:40.822 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:14:46.017 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:14:46.018 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:16:46.221 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:16:46.222 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:18:51.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:18:51.422 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:20:56.593 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:20:56.595 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:23:01.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:23:01.752 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:25:06.937 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:25:06.938 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:27:12.113 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:27:12.114 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:29:17.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:29:17.293 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:31:18.301 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:31:18.302 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:33:21.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:33:21.945 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:35:27.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:35:27.117 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:37:32.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:37:32.294 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:39:36.792 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:39:36.793 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:41:39.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:41:39.489 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:43:44.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:43:44.661 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:45:47.247 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:45:47.248 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:47:52.428 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:47:52.429 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:49:57.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:49:57.609 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:52:02.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:52:02.781 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:54:05.245 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:54:05.246 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:56:10.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:56:10.421 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T04:58:11.217 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T04:58:11.218 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T05:00:16.393 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T05:00:16.395 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T05:02:21.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T05:02:21.569 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T05:04:26.749 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T05:04:26.750 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T05:06:31.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T05:06:31.939 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T05:08:37.115 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T05:08:37.116 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T05:10:42.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-14T05:10:42.294 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-14T05:10:42.369 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-14T05:10:42.535 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-14T05:10:42.536 INFO:teuthology.nuke:Checking targets against current locks 2023-07-14T05:10:42.584 DEBUG:teuthology.nuke:shortname: smithi122 2023-07-14T05:10:42.585 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T05:10:42.609 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-13_23:12:00-rados-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 04:01:36.372916', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJNhYSluTkX2psZp/yv6UgpslPk76nR95WT49pPG4b8UuqQzfKyy9E5WfQpY4S89f9QA767QeuqwF8rE4YSnKG0='} 2023-07-14T05:10:42.612 DEBUG:teuthology.nuke:shortname: smithi151 2023-07-14T05:10:42.612 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-14T05:10:42.637 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-13_23:12:00-rados-wip-yuri2-testing-2023-07-13-1236-distro-default-smithi/7336926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 04:01:36.370836', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMsnlwj2zv0Xmk/1hbFqhjsNSC/+d+zDrXWES7UijzK9cbvWUEVT+9zGMXpRCUNuIKxJfO8s5ig70vfTfiXrpcA='} 2023-07-14T05:10:42.672 INFO:teuthology.orchestra.console:Power off smithi122 2023-07-14T05:10:42.673 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T05:10:42.695 INFO:teuthology.orchestra.console:Power off smithi151 2023-07-14T05:10:42.695 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-14T05:10:42.707 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T05:10:42.721 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-14T05:10:42.833 INFO:teuthology.orchestra.console:Power off for smithi122 completed 2023-07-14T05:10:42.845 INFO:teuthology.orchestra.console:Power off for smithi151 completed 2023-07-14T05:10:42.993 INFO:teuthology.lock.ops:unlocked: smithi122.front.sepia.ceph.com 2023-07-14T05:10:43.001 INFO:teuthology.lock.ops:unlocked: smithi151.front.sepia.ceph.com