2023-07-18T18:23:06.011 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-18T18:23:06.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:23:06.014 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:23:06.156 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3cb36aa8b0>, signals=[15]) 2023-07-18T18:23:06.157 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging 2023-07-18T18:23:06.158 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server 2023-07-18T18:23:06.158 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-07-18T18:23:06.181 INFO:teuthology.lock.ops:Start node 'smithi125.front.sepia.ceph.com' reimaging 2023-07-18T18:23:06.181 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: reset os type and version on server 2023-07-18T18:23:06.181 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-07-18T18:23:06.183 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /host {"name": "smithi027"} 2023-07-18T18:23:06.205 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /host {"name": "smithi125"} 2023-07-18T18:23:06.249 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-18T18:23:06.277 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-18T18:23:06.291 DEBUG:teuthology.provision.fog.smithi027:Requesting image smithi_centos_9.stream (ID 10) 2023-07-18T18:23:06.292 DEBUG:teuthology.provision.fog.smithi027:FOG request: PUT /host/132 {"imageID": 10} 2023-07-18T18:23:06.314 DEBUG:teuthology.provision.fog.smithi125:Requesting image smithi_centos_9.stream (ID 10) 2023-07-18T18:23:06.314 DEBUG:teuthology.provision.fog.smithi125:FOG request: PUT /host/230 {"imageID": 10} 2023-07-18T18:23:06.345 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 9.stream 2023-07-18T18:23:06.345 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:23:06.364 INFO:teuthology.provision.fog.smithi125:Scheduling deploy of centos 9.stream 2023-07-18T18:23:06.364 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:23:06.387 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /tasktype {"name": "deploy"} 2023-07-18T18:23:06.411 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /tasktype {"name": "deploy"} 2023-07-18T18:23:06.423 DEBUG:teuthology.provision.fog.smithi027:FOG request: POST /host/132/task {"taskTypeID": 1} 2023-07-18T18:23:06.445 DEBUG:teuthology.provision.fog.smithi125:FOG request: POST /host/230/task {"taskTypeID": 1} 2023-07-18T18:23:06.507 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:23:06.509 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:23:06.582 INFO:teuthology.orchestra.console:Power off smithi125 2023-07-18T18:23:06.583 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T18:23:06.596 INFO:teuthology.orchestra.console:Power off smithi027 2023-07-18T18:23:06.596 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T18:23:06.604 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:06.616 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:06.725 INFO:teuthology.orchestra.console:Power off for smithi125 completed 2023-07-18T18:23:06.736 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2023-07-18T18:23:06.826 INFO:teuthology.orchestra.console:Power on smithi125 2023-07-18T18:23:06.827 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-18T18:23:06.837 INFO:teuthology.orchestra.console:Power on smithi027 2023-07-18T18:23:06.838 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-18T18:23:06.850 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:06.859 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:10.877 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:10.886 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:15.989 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:15.997 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:21.100 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:21.109 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T18:23:21.312 INFO:teuthology.orchestra.console:Power on for smithi125 completed 2023-07-18T18:23:21.413 INFO:teuthology.provision.fog.smithi125:Waiting for deploy to finish 2023-07-18T18:23:21.413 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:23:26.522 INFO:teuthology.orchestra.console:Power on for smithi027 completed 2023-07-18T18:23:26.622 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish 2023-07-18T18:23:26.623 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:23:41.494 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:23:46.588 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:23:56.565 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:24:01.659 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:24:12.360 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:24:21.740 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:24:27.434 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:24:41.816 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:24:46.824 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:25:01.980 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:25:01.982 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:25:32.063 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:25:32.064 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:25:47.141 DEBUG:teuthology.provision.fog.smithi125:FOG request: GET /task/active None 2023-07-18T18:25:47.144 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:25:47.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:02.212 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:26:05.552 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-18T18:26:11.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:14.608 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-18T18:26:17.264 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-18T18:26:17.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:20.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:23.664 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-18T18:26:29.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:32.752 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-18T18:26:35.760 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-18T18:26:38.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:41.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:41.808 WARNING:teuthology.provision.fog.smithi125:[Errno None] Unable to connect to port 22 on 172.21.15.125 2023-07-18T18:26:44.816 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-18T18:26:47.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:50.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:26:53.872 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-18T18:26:59.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:27:02.960 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-18T18:27:08.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:27:12.048 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-18T18:27:18.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:27:21.136 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-18T18:27:27.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:27:30.224 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-18T18:27:36.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:27:39.312 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-18T18:27:45.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:27:47.817 WARNING:teuthology.provision.fog.smithi125:timed out 2023-07-18T18:27:53.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:27:54.038 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-18T18:27:54.039 DEBUG:teuthology.orchestra.run.smithi125:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-18T18:27:54.364 INFO:teuthology.provision.fog.smithi125:Node is ready 2023-07-18T18:27:54.365 DEBUG:teuthology.orchestra.run.smithi125:> hostname 2023-07-18T18:27:54.380 INFO:teuthology.orchestra.run.smithi125.stdout:smithi125.front.sepia.ceph.com 2023-07-18T18:27:54.380 DEBUG:teuthology.orchestra.run.smithi125:> grep smithi125.front.sepia.ceph.com /etc/hosts 2023-07-18T18:27:54.435 INFO:teuthology.orchestra.run.smithi125.stdout:172.21.15.125 smithi125.front.sepia.ceph.com smithi125 2023-07-18T18:27:54.436 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2023-07-18T18:27:54.553 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hosts 2023-07-18T18:27:54.581 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/172.21.15.125/172.21.15.125/g' /etc/hosts 2023-07-18T18:27:54.651 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i -e 's/smithi125.front.sepia.ceph.com/smithi125/g' /etc/hostname 2023-07-18T18:27:54.719 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname smithi125 2023-07-18T18:27:54.786 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-07-18T18:27:54.840 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2023-07-18T18:27:54.841 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="9" 2023-07-18T18:27:54.841 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2023-07-18T18:27:54.841 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2023-07-18T18:27:54.841 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="9" 2023-07-18T18:27:54.841 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el9" 2023-07-18T18:27:54.841 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-18T18:27:54.842 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-07-18T18:27:54.842 INFO:teuthology.orchestra.run.smithi125.stdout:LOGO="fedora-logo-icon" 2023-07-18T18:27:54.842 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-18T18:27:54.842 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2023-07-18T18:27:54.842 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T18:27:54.842 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-18T18:27:54.842 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-18T18:27:54.843 INFO:teuthology.provision.fog.smithi125:Deploy complete! 2023-07-18T18:27:54.844 DEBUG:teuthology.parallel:result is None 2023-07-18T18:28:45.355 WARNING:teuthology.provision.fog.smithi027:timed out 2023-07-18T18:28:51.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:28:51.551 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-18T18:28:51.552 DEBUG:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-18T18:28:51.883 INFO:teuthology.provision.fog.smithi027:Node is ready 2023-07-18T18:28:51.884 DEBUG:teuthology.orchestra.run.smithi027:> hostname 2023-07-18T18:28:51.899 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027.front.sepia.ceph.com 2023-07-18T18:28:51.899 DEBUG:teuthology.orchestra.run.smithi027:> grep smithi027.front.sepia.ceph.com /etc/hosts 2023-07-18T18:28:51.954 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.15.27 smithi027.front.sepia.ceph.com smithi027 2023-07-18T18:28:51.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027 2023-07-18T18:28:52.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hosts 2023-07-18T18:28:52.094 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/172.21.15.27/172.21.15.27/g' /etc/hosts 2023-07-18T18:28:52.165 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i -e 's/smithi027.front.sepia.ceph.com/smithi027/g' /etc/hostname 2023-07-18T18:28:52.236 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname smithi027 2023-07-18T18:28:52.306 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2023-07-18T18:28:52.360 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2023-07-18T18:28:52.361 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2023-07-18T18:28:52.361 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2023-07-18T18:28:52.361 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2023-07-18T18:28:52.361 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2023-07-18T18:28:52.361 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2023-07-18T18:28:52.361 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-18T18:28:52.362 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2023-07-18T18:28:52.362 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2023-07-18T18:28:52.362 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-18T18:28:52.362 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2023-07-18T18:28:52.362 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T18:28:52.362 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-18T18:28:52.362 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-18T18:28:52.363 INFO:teuthology.provision.fog.smithi027:Deploy complete! 2023-07-18T18:28:52.363 DEBUG:teuthology.parallel:result is None 2023-07-18T18:28:52.520 INFO:teuthology.lock.ops:Checking smithi027.front.sepia.ceph.com 2023-07-18T18:28:52.521 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-18T18:28:52.551 INFO:teuthology.lock.ops:Checking smithi125.front.sepia.ceph.com 2023-07-18T18:28:52.551 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-18T18:28:52.577 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: set os type and version on server 2023-07-18T18:28:52.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-18T18:28:52.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:28:52.671 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-07-18T18:28:52.737 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-18T18:28:52.738 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2023-07-18T18:28:52.791 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2023-07-18T18:28:52.791 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2023-07-18T18:28:52.845 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2023-07-18T18:28:52.845 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2023-07-18T18:28:52.845 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2023-07-18T18:28:52.846 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2023-07-18T18:28:52.846 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2023-07-18T18:28:52.846 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2023-07-18T18:28:52.846 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-18T18:28:52.846 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2023-07-18T18:28:52.846 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2023-07-18T18:28:52.846 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-18T18:28:52.847 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2023-07-18T18:28:52.847 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T18:28:52.847 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-18T18:28:52.847 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-18T18:28:52.847 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-07-18T18:28:52.867 INFO:teuthology.lock.ops:Updating [smithi125.front.sepia.ceph.com]: set os type and version on server 2023-07-18T18:28:52.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-07-18T18:28:52.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-18T18:28:52.948 DEBUG:teuthology.orchestra.run.smithi125:> true 2023-07-18T18:28:53.118 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2023-07-18T18:28:53.118 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2023-07-18T18:28:53.171 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2023-07-18T18:28:53.172 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2023-07-18T18:28:53.225 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2023-07-18T18:28:53.225 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="9" 2023-07-18T18:28:53.226 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2023-07-18T18:28:53.226 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2023-07-18T18:28:53.226 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="9" 2023-07-18T18:28:53.226 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el9" 2023-07-18T18:28:53.226 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-18T18:28:53.226 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2023-07-18T18:28:53.226 INFO:teuthology.orchestra.run.smithi125.stdout:LOGO="fedora-logo-icon" 2023-07-18T18:28:53.227 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-18T18:28:53.227 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2023-07-18T18:28:53.227 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-18T18:28:53.227 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-18T18:28:53.227 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-18T18:28:53.227 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2023-07-18T18:28:53.251 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:28:53.251 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:28:53.346 INFO:teuthology.dispatcher.supervisor:Running job 7343002 2023-07-18T18:28:53.346 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology -v --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2023-07-18_18:17:38-rados-wip-yuri2-testing-2023-07-15-0802-distro-default-smithi/7343002 --name lflores-2023-07-18_18:17:38-rados-wip-yuri2-testing-2023-07-15-0802-distro-default-smithi --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} -- /home/teuthworker/archive/lflores-2023-07-18_18:17:38-rados-wip-yuri2-testing-2023-07-15-0802-distro-default-smithi/7343002/orig.config.yaml 2023-07-18T18:28:53.349 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/lflores-2023-07-18_18:17:38-rados-wip-yuri2-testing-2023-07-15-0802-distro-default-smithi/7343002 2023-07-18T18:28:53.350 INFO:teuthology.dispatcher.supervisor:Job PID: 23948 2023-07-18T18:28:53.350 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-18T18:30:53.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:30:53.361 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:32:53.433 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:32:53.434 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:34:53.598 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:34:53.599 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:36:53.715 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:36:53.716 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:38:53.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:38:53.886 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:40:53.978 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:40:53.979 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:42:54.154 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:42:54.156 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:44:54.234 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:44:54.235 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:46:54.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:46:54.315 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:48:54.402 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:48:54.404 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:50:54.565 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:50:54.566 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:52:54.678 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:52:54.679 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:54:54.784 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:54:54.785 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:56:54.881 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:56:54.882 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T18:58:55.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T18:58:55.074 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T19:00:55.254 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-18T19:00:55.255 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-18T19:00:55.321 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-18T19:00:55.422 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-18T19:00:55.422 INFO:teuthology.nuke:Checking targets against current locks 2023-07-18T19:00:55.479 DEBUG:teuthology.nuke:shortname: smithi027 2023-07-18T19:00:55.480 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T19:00:55.506 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2023-07-18_18:17:38-rados-wip-yuri2-testing-2023-07-15-0802-distro-default-smithi/7343002', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 18:23:05.231442', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIUHBCR4ONlbOALWl06B9JFUAIq/zDn+LrwLNjV3p/v4HtByiJNbC8Bk0GH/qtGa9KYkqZDQVL4aGFz2RbOezAg='} 2023-07-18T19:00:55.513 DEBUG:teuthology.nuke:shortname: smithi125 2023-07-18T19:00:55.513 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-18T19:00:55.534 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2023-07-18_18:17:38-rados-wip-yuri2-testing-2023-07-15-0802-distro-default-smithi/7343002', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-18 18:23:05.229322', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGFbRbWFaawK1GOTCm5qTRpxq9Dm6vnf8PL+mT3uTGDA2k+nJtQIkdosCs5wVN8RSPupHwrJti2j4ajfkG5L6b8='} 2023-07-18T19:00:55.552 INFO:teuthology.orchestra.console:Power off smithi027 2023-07-18T19:00:55.553 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T19:00:55.573 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T19:00:55.593 INFO:teuthology.orchestra.console:Power off smithi125 2023-07-18T19:00:55.594 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-18T19:00:55.612 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-18T19:00:55.702 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2023-07-18T19:00:55.731 INFO:teuthology.orchestra.console:Power off for smithi125 completed 2023-07-18T19:00:55.862 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com 2023-07-18T19:00:55.878 INFO:teuthology.lock.ops:unlocked: smithi125.front.sepia.ceph.com