2023-07-21T06:36:37.414 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T06:36:37.415 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T06:36:37.416 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T06:36:55.039 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f95e7a62f70>, signals=[15]) 2023-07-21T06:36:55.041 INFO:teuthology.lock.ops:Start node 'smithi031.front.sepia.ceph.com' reimaging 2023-07-21T06:36:55.041 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T06:36:55.041 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-07-21T06:36:55.072 INFO:teuthology.lock.ops:Start node 'smithi144.front.sepia.ceph.com' reimaging 2023-07-21T06:36:55.073 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: reset os type and version on server 2023-07-21T06:36:55.073 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-07-21T06:36:55.075 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /host {"name": "smithi031"} 2023-07-21T06:36:55.097 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /host {"name": "smithi144"} 2023-07-21T06:36:55.143 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-21T06:36:55.159 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-21T06:36:55.193 DEBUG:teuthology.provision.fog.smithi031:Requesting image smithi_centos_9.stream (ID 10) 2023-07-21T06:36:55.193 DEBUG:teuthology.provision.fog.smithi031:FOG request: PUT /host/136 {"imageID": 10} 2023-07-21T06:36:55.197 DEBUG:teuthology.provision.fog.smithi144:Requesting image smithi_centos_9.stream (ID 10) 2023-07-21T06:36:55.198 DEBUG:teuthology.provision.fog.smithi144:FOG request: PUT /host/249 {"imageID": 10} 2023-07-21T06:36:55.245 INFO:teuthology.provision.fog.smithi144:Scheduling deploy of centos 9.stream 2023-07-21T06:36:55.245 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:36:55.249 INFO:teuthology.provision.fog.smithi031:Scheduling deploy of centos 9.stream 2023-07-21T06:36:55.249 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:36:55.321 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T06:36:55.322 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /tasktype {"name": "deploy"} 2023-07-21T06:36:55.357 DEBUG:teuthology.provision.fog.smithi031:FOG request: POST /host/136/task {"taskTypeID": 1} 2023-07-21T06:36:55.372 DEBUG:teuthology.provision.fog.smithi144:FOG request: POST /host/249/task {"taskTypeID": 1} 2023-07-21T06:36:55.402 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:36:55.423 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:36:55.511 INFO:teuthology.orchestra.console:Power off smithi031 2023-07-21T06:36:55.512 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T06:36:55.530 INFO:teuthology.orchestra.console:Power off smithi144 2023-07-21T06:36:55.531 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T06:37:00.566 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:05.544 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:05.579 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:09.568 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:10.689 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:10.902 INFO:teuthology.orchestra.console:Power off for smithi144 completed 2023-07-21T06:37:11.003 INFO:teuthology.orchestra.console:Power on smithi144 2023-07-21T06:37:11.003 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T06:37:11.027 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:14.679 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:15.051 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:15.264 INFO:teuthology.orchestra.console:Power on for smithi144 completed 2023-07-21T06:37:15.366 INFO:teuthology.provision.fog.smithi144:Waiting for deploy to finish 2023-07-21T06:37:15.366 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:37:19.790 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:24.903 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:30.016 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:30.437 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:37:35.547 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2023-07-21T06:37:35.648 INFO:teuthology.orchestra.console:Power on smithi031 2023-07-21T06:37:35.649 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-21T06:37:45.508 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:49.532 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:50.519 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:37:53.645 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:37:53.855 INFO:teuthology.orchestra.console:Power on for smithi031 completed 2023-07-21T06:37:53.956 INFO:teuthology.provision.fog.smithi031:Waiting for deploy to finish 2023-07-21T06:37:53.957 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:38:05.600 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:38:09.036 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:38:20.672 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:38:29.108 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:38:40.744 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:38:49.179 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:38:56.094 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:39:09.260 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:39:16.431 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:39:36.504 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:39:36.505 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:39:51.584 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:39:51.586 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-07-21T06:39:56.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:39:59.728 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-07-21T06:40:05.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:08.784 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-07-21T06:40:11.660 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-07-21T06:40:11.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:14.768 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-07-21T06:40:14.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:17.840 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-07-21T06:40:20.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:25.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:30.784 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-07-21T06:40:33.840 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-07-21T06:40:36.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:41.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:44.308 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-07-21T06:40:44.309 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-07-21T06:40:50.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:50.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:40:53.392 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-07-21T06:40:59.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:41:50.364 WARNING:teuthology.provision.fog.smithi144:timed out 2023-07-21T06:41:56.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:41:56.546 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T06:41:56.547 DEBUG:teuthology.orchestra.run.smithi144:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T06:41:56.841 INFO:teuthology.provision.fog.smithi144:Node is ready 2023-07-21T06:41:56.842 DEBUG:teuthology.orchestra.run.smithi144:> hostname 2023-07-21T06:41:56.856 INFO:teuthology.orchestra.run.smithi144.stdout:smithi144.front.sepia.ceph.com 2023-07-21T06:41:56.856 DEBUG:teuthology.orchestra.run.smithi144:> grep smithi144.front.sepia.ceph.com /etc/hosts 2023-07-21T06:41:56.911 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144 smithi144.front.sepia.ceph.com smithi144 2023-07-21T06:41:56.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname smithi144 2023-07-21T06:41:57.023 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/smithi144.front.sepia.ceph.com/smithi144/g' /etc/hosts 2023-07-21T06:41:57.051 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/172.21.15.144/172.21.15.144/g' /etc/hosts 2023-07-21T06:41:57.118 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/smithi144.front.sepia.ceph.com/smithi144/g' /etc/hostname 2023-07-21T06:41:57.185 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname smithi144 2023-07-21T06:41:57.252 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-07-21T06:41:57.307 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2023-07-21T06:41:57.307 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2023-07-21T06:41:57.307 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2023-07-21T06:41:57.307 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2023-07-21T06:41:57.308 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2023-07-21T06:41:57.308 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2023-07-21T06:41:57.308 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T06:41:57.308 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2023-07-21T06:41:57.308 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2023-07-21T06:41:57.308 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T06:41:57.308 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2023-07-21T06:41:57.308 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T06:41:57.309 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T06:41:57.309 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T06:41:57.309 INFO:teuthology.provision.fog.smithi144:Deploy complete! 2023-07-21T06:41:57.311 DEBUG:teuthology.parallel:result is None 2023-07-21T06:41:59.402 WARNING:teuthology.provision.fog.smithi031:timed out 2023-07-21T06:42:05.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:42:05.636 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-21T06:42:05.636 DEBUG:teuthology.orchestra.run.smithi031:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-21T06:42:05.993 INFO:teuthology.provision.fog.smithi031:Node is ready 2023-07-21T06:42:05.993 DEBUG:teuthology.orchestra.run.smithi031:> hostname 2023-07-21T06:42:06.008 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031.front.sepia.ceph.com 2023-07-21T06:42:06.008 DEBUG:teuthology.orchestra.run.smithi031:> grep smithi031.front.sepia.ceph.com /etc/hosts 2023-07-21T06:42:06.069 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31 smithi031.front.sepia.ceph.com smithi031 2023-07-21T06:42:06.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname smithi031 2023-07-21T06:42:06.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/smithi031.front.sepia.ceph.com/smithi031/g' /etc/hosts 2023-07-21T06:42:06.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/172.21.15.31/172.21.15.31/g' /etc/hosts 2023-07-21T06:42:06.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/smithi031.front.sepia.ceph.com/smithi031/g' /etc/hostname 2023-07-21T06:42:06.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname smithi031 2023-07-21T06:42:06.457 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-07-21T06:42:06.517 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2023-07-21T06:42:06.517 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2023-07-21T06:42:06.517 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2023-07-21T06:42:06.517 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2023-07-21T06:42:06.518 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2023-07-21T06:42:06.518 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2023-07-21T06:42:06.518 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T06:42:06.518 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-07-21T06:42:06.518 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2023-07-21T06:42:06.518 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T06:42:06.519 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2023-07-21T06:42:06.519 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T06:42:06.519 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T06:42:06.519 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T06:42:06.519 INFO:teuthology.provision.fog.smithi031:Deploy complete! 2023-07-21T06:42:06.520 DEBUG:teuthology.parallel:result is None 2023-07-21T06:42:08.947 INFO:teuthology.lock.ops:Checking smithi031.front.sepia.ceph.com 2023-07-21T06:42:08.948 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T06:42:08.970 INFO:teuthology.lock.ops:Checking smithi144.front.sepia.ceph.com 2023-07-21T06:42:08.970 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-21T06:42:08.988 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: set os type and version on server 2023-07-21T06:42:08.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-07-21T06:42:08.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:42:09.089 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-07-21T06:42:09.160 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-07-21T06:42:09.160 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-07-21T06:42:09.220 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-07-21T06:42:09.220 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-07-21T06:42:09.279 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2023-07-21T06:42:09.279 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2023-07-21T06:42:09.280 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2023-07-21T06:42:09.280 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2023-07-21T06:42:09.280 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2023-07-21T06:42:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2023-07-21T06:42:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T06:42:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-07-21T06:42:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2023-07-21T06:42:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T06:42:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2023-07-21T06:42:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T06:42:09.281 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T06:42:09.282 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T06:42:09.282 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-07-21T06:42:09.304 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: set os type and version on server 2023-07-21T06:42:09.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-07-21T06:42:09.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T06:42:09.387 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-07-21T06:42:09.559 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-07-21T06:42:09.559 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2023-07-21T06:42:09.613 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2023-07-21T06:42:09.614 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-07-21T06:42:09.668 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2023-07-21T06:42:09.668 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2023-07-21T06:42:09.668 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2023-07-21T06:42:09.668 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2023-07-21T06:42:09.669 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2023-07-21T06:42:09.669 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2023-07-21T06:42:09.669 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-21T06:42:09.669 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2023-07-21T06:42:09.669 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2023-07-21T06:42:09.669 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-21T06:42:09.669 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2023-07-21T06:42:09.670 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T06:42:09.670 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-21T06:42:09.670 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-21T06:42:09.670 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-07-21T06:42:09.696 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T06:42:09.697 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T06:42:09.788 INFO:teuthology.dispatcher.supervisor:Running job 7346363 2023-07-21T06:42:09.788 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346363 --name vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-ec-root overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/fs/snaps}} -- /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346363/orig.config.yaml 2023-07-21T06:42:09.795 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346363 2023-07-21T06:42:09.796 INFO:teuthology.dispatcher.supervisor:Job PID: 21243 2023-07-21T06:42:09.797 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-21T06:44:09.898 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T06:44:09.899 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-21T06:44:14.057 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-21T06:44:14.337 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-21T06:44:14.338 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T06:44:14.395 DEBUG:teuthology.nuke:shortname: smithi031 2023-07-21T06:44:14.396 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T06:44:14.409 DEBUG:teuthology.nuke:shortname: smithi144 2023-07-21T06:44:14.410 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T06:44:14.420 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346363', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 06:36:21.620630', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQPv93xJJeJ61PGi0ivwYcn5mWDgU3sU22O43c+kx0i9o1nzABZi1Z6IZCTXxzlPulrgzwdPOpKOgSz3EEkaTQ='} 2023-07-21T06:44:14.430 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-07-21_06:30:09-fs-wip-vshankar-testing-20230720.042515-testing-default-smithi/7346363', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 06:36:21.618823', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHU7Ye7jFIt7sOK3UJDv6fBOOOBANDL3003t9iy4k3nlLqG5mF6u1ApAGO5S45c81dntuAUw/YJWbg9QGOZnF7g='} 2023-07-21T06:44:14.469 INFO:teuthology.orchestra.console:Power off smithi031 2023-07-21T06:44:14.469 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T06:44:14.485 INFO:teuthology.orchestra.console:Power off smithi144 2023-07-21T06:44:14.486 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T06:44:14.495 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:44:14.512 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T06:44:14.624 INFO:teuthology.orchestra.console:Power off for smithi031 completed 2023-07-21T06:44:14.634 INFO:teuthology.orchestra.console:Power off for smithi144 completed 2023-07-21T06:44:14.780 INFO:teuthology.lock.ops:unlocked: smithi031.front.sepia.ceph.com 2023-07-21T06:44:14.789 INFO:teuthology.lock.ops:unlocked: smithi144.front.sepia.ceph.com