2023-10-14T01:50:58.569 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-14T01:50:58.570 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T01:50:58.706 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2ecc5b1670>, signals=[15]) 2023-10-14T01:50:58.707 INFO:teuthology.lock.ops:Start node 'smithi070.front.sepia.ceph.com' reimaging 2023-10-14T01:50:58.708 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: reset os type and version on server 2023-10-14T01:50:58.708 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2023-10-14T01:50:58.739 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging 2023-10-14T01:50:58.739 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server 2023-10-14T01:50:58.739 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-10-14T01:50:58.741 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /host {"name": "smithi070"} 2023-10-14T01:50:58.764 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /host {"name": "smithi125"} 2023-10-14T01:50:58.816 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-14T01:50:58.842 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-10-14T01:50:58.858 DEBUG:teuthology.provision.fog.smithi070:Requesting image smithi_centos_8.stream (ID 3) 2023-10-14T01:50:58.859 DEBUG:teuthology.provision.fog.smithi070:FOG request: PUT /host/175 {"imageID": 3} 2023-10-14T01:50:58.886 DEBUG:teuthology.provision.fog.smithi125:Requesting image smithi_centos_8.stream (ID 3) 2023-10-14T01:50:58.887 DEBUG:teuthology.provision.fog.smithi125:FOG request: PUT /host/230 {"imageID": 3} 2023-10-14T01:50:58.905 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of centos 8.stream 2023-10-14T01:50:58.905 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:50:58.934 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of centos 8.stream 2023-10-14T01:50:58.935 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:50:58.960 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /tasktype {"name": "deploy"} 2023-10-14T01:50:58.981 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /tasktype {"name": "deploy"} 2023-10-14T01:50:59.005 DEBUG:teuthology.provision.fog.smithi070:FOG request: POST /host/175/task {"taskTypeID": 1} 2023-10-14T01:50:59.018 DEBUG:teuthology.provision.fog.smithi125:FOG request: POST /host/230/task {"taskTypeID": 1} 2023-10-14T01:50:59.072 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:50:59.088 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:50:59.161 INFO:teuthology.orchestra.console.smithi070:Power off 2023-10-14T01:50:59.162 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T01:50:59.177 INFO:teuthology.orchestra.console.smithi125:Power off 2023-10-14T01:50:59.178 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T01:50:59.188 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2023-10-14T01:50:59.189 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:50:59.204 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2023-10-14T01:50:59.205 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:50:59.213 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-10-14T01:50:59.232 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-14T01:51:03.218 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:03.233 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:03.327 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-10-14T01:51:03.342 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-14T01:51:07.330 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:07.343 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:07.439 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2023-10-14T01:51:07.452 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-14T01:51:07.540 INFO:teuthology.orchestra.console.smithi070:Power off completed 2023-10-14T01:51:07.641 INFO:teuthology.orchestra.console.smithi070:Power on 2023-10-14T01:51:07.641 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-14T01:51:07.662 DEBUG:teuthology.orchestra.console.smithi070:power on output: Chassis Power Control: Up/On 2023-10-14T01:51:07.663 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:07.686 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2023-10-14T01:51:11.454 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:11.563 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-14T01:51:11.664 INFO:teuthology.orchestra.console.smithi125:Power off completed 2023-10-14T01:51:11.686 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:11.765 INFO:teuthology.orchestra.console.smithi125:Power on 2023-10-14T01:51:11.765 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-14T01:51:11.788 DEBUG:teuthology.orchestra.console.smithi125:power on output: Chassis Power Control: Up/On 2023-10-14T01:51:11.789 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:11.797 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2023-10-14T01:51:11.810 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-14T01:51:15.802 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:15.811 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:15.919 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-14T01:51:16.029 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-10-14T01:51:16.130 INFO:teuthology.orchestra.console.smithi070:Power on completed 2023-10-14T01:51:16.231 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish 2023-10-14T01:51:16.232 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:51:19.922 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T01:51:20.031 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2023-10-14T01:51:20.132 INFO:teuthology.orchestra.console.smithi125:Power on completed 2023-10-14T01:51:20.233 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish 2023-10-14T01:51:20.234 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:51:31.314 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:51:35.308 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:51:46.388 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:51:50.370 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:52:01.458 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:52:05.426 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:52:16.524 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:52:20.490 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:52:31.598 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:52:35.546 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:52:46.678 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:52:50.615 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:53:01.747 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:53:05.674 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:53:16.822 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:53:20.749 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:53:31.892 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:53:35.818 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:53:46.961 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:53:50.898 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-10-14T01:53:50.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:02.037 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None 2023-10-14T01:54:02.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:09.406 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-14T01:54:15.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:18.497 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-14T01:54:20.414 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-10-14T01:54:24.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:26.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:27.578 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-14T01:54:29.498 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-10-14T01:54:33.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:35.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:36.634 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-14T01:54:38.558 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-10-14T01:54:42.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:44.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:45.722 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-10-14T01:54:47.646 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-10-14T01:54:51.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:54:53.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:55:51.786 WARNING:teuthology.provision.fog.smithi125:timed out 2023-10-14T01:55:53.654 WARNING:teuthology.provision.fog.smithi070:timed out 2023-10-14T01:55:57.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:55:58.070 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-14T01:55:58.071 DEBUG:teuthology.orchestra.run.smithi125:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-14T01:55:58.325 INFO:teuthology.provision.fog.smithi125:Node is ready 2023-10-14T01:55:58.326 DEBUG:teuthology.orchestra.run.smithi125:> hostname 2023-10-14T01:55:58.347 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com 2023-10-14T01:55:58.348 DEBUG:teuthology.orchestra.run.smithi125:> grep smithi125.front.sepia.ceph.com /etc/hosts 2023-10-14T01:55:58.409 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2023-10-14T01:55:58.410 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2023-10-14T01:55:58.515 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hosts 2023-10-14T01:55:58.550 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/172.21.15.125/172.21.15.125/g' /etc/hosts 2023-10-14T01:55:58.624 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hostname 2023-10-14T01:55:58.698 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2023-10-14T01:55:58.771 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-10-14T01:55:58.833 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2023-10-14T01:55:58.834 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="8" 2023-10-14T01:55:58.834 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2023-10-14T01:55:58.834 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2023-10-14T01:55:58.834 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="8" 2023-10-14T01:55:58.835 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el8" 2023-10-14T01:55:58.835 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-14T01:55:58.835 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-10-14T01:55:58.835 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-14T01:55:58.835 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2023-10-14T01:55:58.836 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T01:55:58.836 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T01:55:58.836 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-14T01:55:58.837 INFO:teuthology.provision.fog.smithi125:Deploy complete! 2023-10-14T01:55:58.839 DEBUG:teuthology.parallel:result is None 2023-10-14T01:55:59.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:55:59.896 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-14T01:55:59.896 DEBUG:teuthology.orchestra.run.smithi070:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-14T01:56:00.149 INFO:teuthology.provision.fog.smithi070:Node is ready 2023-10-14T01:56:00.149 DEBUG:teuthology.orchestra.run.smithi070:> hostname 2023-10-14T01:56:00.168 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070.front.sepia.ceph.com 2023-10-14T01:56:00.169 DEBUG:teuthology.orchestra.run.smithi070:> grep smithi070.front.sepia.ceph.com /etc/hosts 2023-10-14T01:56:00.227 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70 smithi070.front.sepia.ceph.com smithi070 2023-10-14T01:56:00.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname smithi070 2023-10-14T01:56:00.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/smithi070.front.sepia.ceph.com/smithi070/g' /etc/hosts 2023-10-14T01:56:00.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/172.21.15.70/172.21.15.70/g' /etc/hosts 2023-10-14T01:56:00.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/smithi070.front.sepia.ceph.com/smithi070/g' /etc/hostname 2023-10-14T01:56:00.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname smithi070 2023-10-14T01:56:00.568 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2023-10-14T01:56:00.625 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2023-10-14T01:56:00.625 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8" 2023-10-14T01:56:00.626 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2023-10-14T01:56:00.626 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2023-10-14T01:56:00.626 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8" 2023-10-14T01:56:00.626 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8" 2023-10-14T01:56:00.626 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-14T01:56:00.627 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2023-10-14T01:56:00.627 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-14T01:56:00.627 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2023-10-14T01:56:00.627 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T01:56:00.627 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T01:56:00.628 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-14T01:56:00.628 INFO:teuthology.provision.fog.smithi070:Deploy complete! 2023-10-14T01:56:00.629 DEBUG:teuthology.parallel:result is None 2023-10-14T01:56:00.777 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com 2023-10-14T01:56:00.778 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-14T01:56:00.808 INFO:teuthology.lock.ops:Checking smithi125.front.sepia.ceph.com 2023-10-14T01:56:00.808 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-14T01:56:00.834 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server 2023-10-14T01:56:00.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-14T01:56:00.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:56:00.902 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-10-14T01:56:00.968 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-14T01:56:00.968 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2023-10-14T01:56:01.026 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2023-10-14T01:56:01.026 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2023-10-14T01:56:01.084 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2023-10-14T01:56:01.084 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8" 2023-10-14T01:56:01.084 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2023-10-14T01:56:01.084 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2023-10-14T01:56:01.085 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8" 2023-10-14T01:56:01.085 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8" 2023-10-14T01:56:01.085 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-14T01:56:01.085 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2023-10-14T01:56:01.086 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-14T01:56:01.086 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2023-10-14T01:56:01.086 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T01:56:01.086 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T01:56:01.086 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-14T01:56:01.087 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2023-10-14T01:56:01.108 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: set os type and version on server 2023-10-14T01:56:01.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-14T01:56:01.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-14T01:56:01.177 DEBUG:teuthology.orchestra.run.smithi125:> true 2023-10-14T01:56:01.252 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-10-14T01:56:01.253 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2023-10-14T01:56:01.314 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2023-10-14T01:56:01.315 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-10-14T01:56:01.376 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2023-10-14T01:56:01.377 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="8" 2023-10-14T01:56:01.377 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2023-10-14T01:56:01.377 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2023-10-14T01:56:01.377 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="8" 2023-10-14T01:56:01.378 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el8" 2023-10-14T01:56:01.378 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-14T01:56:01.378 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-10-14T01:56:01.378 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-14T01:56:01.378 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2023-10-14T01:56:01.379 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-14T01:56:01.379 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-14T01:56:01.379 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-14T01:56:01.379 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-10-14T01:56:01.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T01:56:01.492 INFO:teuthology.dispatcher.supervisor:Running job 7424791 2023-10-14T01:56:01.493 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-13_15:02:02-rgw-wip-neorados-learning-from-experience-distro-default-smithi/7424791 --name yuriw-2023-10-13_15:02:02-rgw-wip-neorados-learning-from-experience-distro-default-smithi --description rgw/upgrade/{1-install/pacific/{distro$/{centos_8.stream} install overrides} 2-setup 3-upgrade-sequence/osds-then-rgws cluster frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides} -- /home/teuthworker/archive/yuriw-2023-10-13_15:02:02-rgw-wip-neorados-learning-from-experience-distro-default-smithi/7424791/orig.config.yaml 2023-10-14T01:56:01.496 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-10-13_15:02:02-rgw-wip-neorados-learning-from-experience-distro-default-smithi/7424791 2023-10-14T01:56:01.497 INFO:teuthology.dispatcher.supervisor:Job PID: 2159387 2023-10-14T01:56:01.497 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-14T01:58:01.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:00:01.767 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:02:01.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:04:02.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:06:02.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:08:02.423 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:10:02.591 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:12:02.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:14:02.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:16:03.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:18:03.275 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:20:03.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:22:03.607 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:24:03.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:24:03.832 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-14T02:24:03.997 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-14T02:24:03.998 INFO:teuthology.nuke:Checking targets against current locks 2023-10-14T02:24:04.049 DEBUG:teuthology.nuke:shortname: smithi070 2023-10-14T02:24:04.049 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T02:24:04.068 DEBUG:teuthology.nuke:shortname: smithi125 2023-10-14T02:24:04.068 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T02:24:04.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-13_15:02:02-rgw-wip-neorados-learning-from-experience-distro-default-smithi/7424791', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 01:50:57.711190', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKIg0qd3il1NImOsauD00GZhgdlTbkMc45CAxOkretVFnV0di0oE1LtoOdlTSSWqZYhOT6EpGdyeOENXdhRmmrk='} 2023-10-14T02:24:04.095 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-13_15:02:02-rgw-wip-neorados-learning-from-experience-distro-default-smithi/7424791', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-14 01:50:57.709255', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9zO4PlK1AK8kwYGtbzYMO7bBKDr2scv5cHVhPXhldM/lZG85KT1oVJ1hsy5ai8RyfwRb38kF+W0BfQ2DTC4mQ='} 2023-10-14T02:24:04.121 INFO:teuthology.orchestra.console.smithi070:Power off 2023-10-14T02:24:04.121 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T02:24:04.142 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2023-10-14T02:24:04.143 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T02:24:04.159 INFO:teuthology.orchestra.console.smithi125:Power off 2023-10-14T02:24:04.160 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T02:24:04.167 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2023-10-14T02:24:04.178 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2023-10-14T02:24:04.179 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T02:24:04.202 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2023-10-14T02:24:04.269 INFO:teuthology.orchestra.console.smithi070:Power off completed 2023-10-14T02:24:04.304 INFO:teuthology.orchestra.console.smithi125:Power off completed 2023-10-14T02:24:04.423 INFO:teuthology.lock.ops:unlocked: smithi070.front.sepia.ceph.com 2023-10-14T02:24:04.456 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com