2023-10-21T10:08:56.021 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-21T10:08:56.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:08:56.165 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb58e795670>, signals=[15]) 2023-10-21T10:08:56.166 INFO:teuthology.lock.ops:Start node 'smithi031.front.sepia.ceph.com' reimaging 2023-10-21T10:08:56.166 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: reset os type and version on server 2023-10-21T10:08:56.167 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-10-21T10:08:56.194 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging 2023-10-21T10:08:56.195 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server 2023-10-21T10:08:56.195 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-10-21T10:08:56.198 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /host {"name": "smithi031"} 2023-10-21T10:08:56.219 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /host {"name": "smithi125"} 2023-10-21T10:08:56.275 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-10-21T10:08:56.297 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-10-21T10:08:56.314 DEBUG:teuthology.provision.fog.smithi031:Requesting image smithi_centos_9.stream (ID 10) 2023-10-21T10:08:56.315 DEBUG:teuthology.provision.fog.smithi031:FOG request: PUT /host/136 {"imageID": 10} 2023-10-21T10:08:56.340 DEBUG:teuthology.provision.fog.smithi125:Requesting image smithi_centos_9.stream (ID 10) 2023-10-21T10:08:56.341 DEBUG:teuthology.provision.fog.smithi125:FOG request: PUT /host/230 {"imageID": 10} 2023-10-21T10:08:56.370 INFO:teuthology.provision.fog.smithi031:Scheduling deploy of centos 9.stream 2023-10-21T10:08:56.370 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:08:56.388 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of centos 9.stream 2023-10-21T10:08:56.388 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:08:56.433 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /tasktype {"name": "deploy"} 2023-10-21T10:08:56.442 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /tasktype {"name": "deploy"} 2023-10-21T10:08:56.471 DEBUG:teuthology.provision.fog.smithi031:FOG request: POST /host/136/task {"taskTypeID": 1} 2023-10-21T10:08:56.477 DEBUG:teuthology.provision.fog.smithi125:FOG request: POST /host/230/task {"taskTypeID": 1} 2023-10-21T10:08:56.527 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:08:56.530 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:08:56.635 INFO:teuthology.orchestra.console.smithi125:Power off 2023-10-21T10:08:56.636 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T10:08:56.649 INFO:teuthology.orchestra.console.smithi031:Power off 2023-10-21T10:08:56.650 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T10:08:56.657 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2023-10-21T10:08:56.658 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T10:08:56.670 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-10-21T10:08:56.671 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T10:08:56.679 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-21T10:08:56.693 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-10-21T10:08:56.780 INFO:teuthology.orchestra.console.smithi125:Power off completed 2023-10-21T10:08:56.794 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-10-21T10:08:56.882 INFO:teuthology.orchestra.console.smithi125:Power on 2023-10-21T10:08:56.882 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-21T10:08:56.895 INFO:teuthology.orchestra.console.smithi031:Power on 2023-10-21T10:08:56.896 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-21T10:08:56.904 DEBUG:teuthology.orchestra.console.smithi125:power on output: Chassis Power Control: Up/On 2023-10-21T10:08:56.905 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T10:08:56.917 DEBUG:teuthology.orchestra.console.smithi031:power on output: Chassis Power Control: Up/On 2023-10-21T10:08:56.918 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T10:08:56.928 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-21T10:08:56.940 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-10-21T10:09:00.930 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T10:09:00.941 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T10:09:01.040 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-21T10:09:01.050 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-10-21T10:09:05.044 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T10:09:05.051 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T10:09:05.153 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-21T10:09:05.161 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-10-21T10:09:05.254 INFO:teuthology.orchestra.console.smithi125:Power on completed 2023-10-21T10:09:05.262 INFO:teuthology.orchestra.console.smithi031:Power on completed 2023-10-21T10:09:05.356 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish 2023-10-21T10:09:05.356 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:09:05.363 INFO:teuthology.provision.fog.smithi031:Waiting for deploy to finish 2023-10-21T10:09:05.364 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:09:20.443 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:09:20.445 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:09:35.534 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:09:35.536 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:09:50.618 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:09:50.620 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:10:05.706 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:10:05.708 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:10:20.798 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:10:20.800 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:10:35.898 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:10:35.900 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:10:50.987 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:10:50.989 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:11:06.078 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:11:06.080 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:11:21.166 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:11:21.168 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-10-21T10:11:21.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:11:36.266 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-21T10:11:36.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:11:39.774 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-21T10:11:45.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:11:48.858 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-21T10:11:54.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:11:54.879 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-21T10:11:57.946 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-21T10:12:00.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:12:03.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:12:03.962 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-21T10:12:07.002 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-10-21T10:12:09.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:12:13.006 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:12:13.018 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-21T10:12:19.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:12:22.078 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-21T10:12:28.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:12:31.162 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-21T10:12:37.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:12:55.546 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-21T10:13:01.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:13:04.606 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-21T10:13:10.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:13:13.013 WARNING:teuthology.provision.fog.smithi031:timed out 2023-10-21T10:13:13.690 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-21T10:13:19.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:13:19.289 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-21T10:13:19.290 DEBUG:teuthology.orchestra.run.smithi031:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-21T10:13:19.628 INFO:teuthology.provision.fog.smithi031:Node is ready 2023-10-21T10:13:19.628 DEBUG:teuthology.orchestra.run.smithi031:> hostname 2023-10-21T10:13:19.644 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031.front.sepia.ceph.com 2023-10-21T10:13:19.644 DEBUG:teuthology.orchestra.run.smithi031:> grep smithi031.front.sepia.ceph.com /etc/hosts 2023-10-21T10:13:19.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:13:19.700 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31 smithi031.front.sepia.ceph.com smithi031 2023-10-21T10:13:19.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname smithi031 2023-10-21T10:13:19.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/smithi031.front.sepia.ceph.com/smithi031/g' /etc/hosts 2023-10-21T10:13:19.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/172.21.15.31/172.21.15.31/g' /etc/hosts 2023-10-21T10:13:19.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/smithi031.front.sepia.ceph.com/smithi031/g' /etc/hostname 2023-10-21T10:13:19.966 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-21T10:13:19.967 DEBUG:teuthology.orchestra.run.smithi125:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-21T10:13:19.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname smithi031 2023-10-21T10:13:20.049 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-10-21T10:13:20.104 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2023-10-21T10:13:20.104 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2023-10-21T10:13:20.104 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2023-10-21T10:13:20.105 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2023-10-21T10:13:20.105 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2023-10-21T10:13:20.105 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2023-10-21T10:13:20.105 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T10:13:20.105 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-10-21T10:13:20.106 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2023-10-21T10:13:20.106 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T10:13:20.106 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2023-10-21T10:13:20.106 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T10:13:20.106 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T10:13:20.107 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T10:13:20.107 INFO:teuthology.provision.fog.smithi031:Deploy complete! 2023-10-21T10:13:20.109 DEBUG:teuthology.parallel:result is None 2023-10-21T10:13:35.305 INFO:teuthology.provision.fog.smithi125:Node is ready 2023-10-21T10:13:35.306 DEBUG:teuthology.orchestra.run.smithi125:> hostname 2023-10-21T10:13:35.365 INFO:teuthology.orchestra.run.smithi125.stdout:smithi046 2023-10-21T10:13:35.366 DEBUG:teuthology.orchestra.run.smithi125:> grep smithi046 /etc/hosts 2023-10-21T10:13:35.423 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2023-10-21T10:13:35.424 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2023-10-21T10:13:35.554 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi046/smithi125/g' /etc/hosts 2023-10-21T10:13:35.585 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/172.21.15.46/172.21.15.125/g' /etc/hosts 2023-10-21T10:13:35.654 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi046/smithi125/g' /etc/hostname 2023-10-21T10:13:35.723 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2023-10-21T10:13:35.791 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-10-21T10:13:35.847 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2023-10-21T10:13:35.848 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="9" 2023-10-21T10:13:35.848 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2023-10-21T10:13:35.848 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2023-10-21T10:13:35.848 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="9" 2023-10-21T10:13:35.849 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el9" 2023-10-21T10:13:35.849 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T10:13:35.849 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-10-21T10:13:35.849 INFO:teuthology.orchestra.run.smithi125.stdout:LOGO="fedora-logo-icon" 2023-10-21T10:13:35.849 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T10:13:35.850 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2023-10-21T10:13:35.850 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T10:13:35.850 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T10:13:35.850 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T10:13:35.851 INFO:teuthology.provision.fog.smithi125:Deploy complete! 2023-10-21T10:13:35.852 DEBUG:teuthology.parallel:result is None 2023-10-21T10:13:36.026 INFO:teuthology.lock.ops:Checking smithi031.front.sepia.ceph.com 2023-10-21T10:13:36.027 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-21T10:13:36.047 INFO:teuthology.lock.ops:Checking smithi125.front.sepia.ceph.com 2023-10-21T10:13:36.047 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-21T10:13:36.073 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: set os type and version on server 2023-10-21T10:13:36.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-21T10:13:36.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:13:36.171 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-10-21T10:13:36.353 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-10-21T10:13:36.353 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-10-21T10:13:36.408 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-10-21T10:13:36.408 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-10-21T10:13:36.462 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2023-10-21T10:13:36.463 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2023-10-21T10:13:36.463 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2023-10-21T10:13:36.463 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2023-10-21T10:13:36.463 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2023-10-21T10:13:36.464 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2023-10-21T10:13:36.464 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T10:13:36.464 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-10-21T10:13:36.464 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2023-10-21T10:13:36.464 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T10:13:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2023-10-21T10:13:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T10:13:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T10:13:36.465 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T10:13:36.466 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-10-21T10:13:36.489 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: set os type and version on server 2023-10-21T10:13:36.489 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-21T10:13:36.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-21T10:13:36.573 DEBUG:teuthology.orchestra.run.smithi125:> true 2023-10-21T10:13:36.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-21T10:13:36.645 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2023-10-21T10:13:36.700 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2023-10-21T10:13:36.700 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-10-21T10:13:36.755 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2023-10-21T10:13:36.755 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="9" 2023-10-21T10:13:36.755 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2023-10-21T10:13:36.756 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2023-10-21T10:13:36.756 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="9" 2023-10-21T10:13:36.756 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el9" 2023-10-21T10:13:36.756 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-21T10:13:36.756 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-10-21T10:13:36.757 INFO:teuthology.orchestra.run.smithi125.stdout:LOGO="fedora-logo-icon" 2023-10-21T10:13:36.757 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-21T10:13:36.757 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2023-10-21T10:13:36.757 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-21T10:13:36.758 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-21T10:13:36.758 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-21T10:13:36.759 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-10-21T10:13:36.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:13:36.885 INFO:teuthology.dispatcher.supervisor:Running job 7433889 2023-10-21T10:13:36.885 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_202b180cb047e798fb131047314a862593f45403/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-20_18:40:51-rados-main-distro-default-smithi/7433889 --name yuriw-2023-10-20_18:40:51-rados-main-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} -- /home/teuthworker/archive/yuriw-2023-10-20_18:40:51-rados-main-distro-default-smithi/7433889/orig.config.yaml 2023-10-21T10:13:36.889 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-10-20_18:40:51-rados-main-distro-default-smithi/7433889 2023-10-21T10:13:36.891 INFO:teuthology.dispatcher.supervisor:Job PID: 3006103 2023-10-21T10:13:36.891 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-21T10:15:36.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:17:37.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:19:37.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:21:37.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:23:37.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:25:37.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:27:38.071 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:29:38.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:31:38.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:33:38.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:35:38.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:37:38.886 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:39:39.051 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:41:39.222 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:43:39.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:45:39.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:47:39.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:49:39.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:51:40.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:53:40.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:55:40.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:57:40.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T10:59:40.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:01:40.867 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:03:41.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:05:41.203 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:07:41.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:09:41.533 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:11:41.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:13:41.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:15:42.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:17:42.190 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:19:42.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:21:42.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:23:42.673 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:25:42.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:27:43.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:29:43.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:31:43.340 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:33:43.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-21T11:33:43.578 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-21T11:33:43.713 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-21T11:33:43.714 INFO:teuthology.nuke:Checking targets against current locks 2023-10-21T11:33:43.771 DEBUG:teuthology.nuke:shortname: smithi031 2023-10-21T11:33:43.772 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T11:33:43.793 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-20_18:40:51-rados-main-distro-default-smithi/7433889', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 10:08:55.204928', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIHtYoRX1aqZvOO1wDZbUcft92+zGng96QMdcKvkeKLjC/ysiKP9aKfZ5I/dPB6xKJ/4tI/FKcHxgOHSYcOrkoU='} 2023-10-21T11:33:43.797 DEBUG:teuthology.nuke:shortname: smithi125 2023-10-21T11:33:43.797 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-21T11:33:43.819 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-20_18:40:51-rados-main-distro-default-smithi/7433889', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-21 10:08:55.206992', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGDIEzWDebOZMmVPqiqz3fAYPtdRqkX7wLbb/9hgSJsGUsBu7PsN6C4wxI5sus5rfeeWCFYF4VAExPnERyzMme0='} 2023-10-21T11:33:43.849 INFO:teuthology.orchestra.console.smithi031:Power off 2023-10-21T11:33:43.849 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T11:33:43.871 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-10-21T11:33:43.871 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T11:33:43.885 INFO:teuthology.orchestra.console.smithi125:Power off 2023-10-21T11:33:43.886 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-21T11:33:43.895 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-10-21T11:33:43.906 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2023-10-21T11:33:43.907 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-21T11:33:43.927 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-21T11:33:43.996 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-10-21T11:33:44.029 INFO:teuthology.orchestra.console.smithi125:Power off completed 2023-10-21T11:33:44.154 INFO:teuthology.lock.ops:unlocked: smithi031.front.sepia.ceph.com 2023-10-21T11:33:44.177 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com