2023-08-30T18:35:30.327 INFO:root:teuthology version: 0.0.1.dev192+gf9a8556 2023-08-30T18:35:30.330 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-30T18:35:30.496 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0a6f0b7670>, signals=[15]) 2023-08-30T18:35:30.497 INFO:teuthology.lock.ops:Start node 'smithi123.front.sepia.ceph.com' reimaging 2023-08-30T18:35:30.497 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: reset os type and version on server 2023-08-30T18:35:30.498 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-08-30T18:35:30.517 INFO:teuthology.lock.ops:Start node 'smithi141.front.sepia.ceph.com' reimaging 2023-08-30T18:35:30.518 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: reset os type and version on server 2023-08-30T18:35:30.518 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2023-08-30T18:35:30.519 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /host {"name": "smithi123"} 2023-08-30T18:35:30.542 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /host {"name": "smithi141"} 2023-08-30T18:35:30.607 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-08-30T18:35:30.617 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-08-30T18:35:30.650 DEBUG:teuthology.provision.fog.smithi123:Requesting image smithi_centos_9.stream (ID 10) 2023-08-30T18:35:30.650 DEBUG:teuthology.provision.fog.smithi123:FOG request: PUT /host/228 {"imageID": 10} 2023-08-30T18:35:30.658 DEBUG:teuthology.provision.fog.smithi141:Requesting image smithi_centos_9.stream (ID 10) 2023-08-30T18:35:30.658 DEBUG:teuthology.provision.fog.smithi141:FOG request: PUT /host/246 {"imageID": 10} 2023-08-30T18:35:30.694 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of centos 9.stream 2023-08-30T18:35:30.694 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:35:30.701 INFO:teuthology.provision.fog.smithi141:Scheduling deploy of centos 9.stream 2023-08-30T18:35:30.701 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:35:30.764 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /tasktype {"name": "deploy"} 2023-08-30T18:35:30.769 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /tasktype {"name": "deploy"} 2023-08-30T18:35:30.819 DEBUG:teuthology.provision.fog.smithi141:FOG request: POST /host/246/task {"taskTypeID": 1} 2023-08-30T18:35:30.820 DEBUG:teuthology.provision.fog.smithi123:FOG request: POST /host/228/task {"taskTypeID": 1} 2023-08-30T18:35:30.884 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:35:30.888 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:35:30.996 INFO:teuthology.orchestra.console.smithi123:Power off 2023-08-30T18:35:30.997 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-30T18:35:31.013 INFO:teuthology.orchestra.console.smithi141:Power off 2023-08-30T18:35:31.014 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-30T18:35:31.022 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-08-30T18:35:31.023 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:35:31.035 DEBUG:teuthology.orchestra.console.smithi141:power off output: Chassis Power Control: Down/Off 2023-08-30T18:35:31.036 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:35:31.045 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-08-30T18:35:31.057 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-08-30T18:35:31.146 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-08-30T18:35:31.158 INFO:teuthology.orchestra.console.smithi141:Power off completed 2023-08-30T18:35:31.247 INFO:teuthology.orchestra.console.smithi123:Power on 2023-08-30T18:35:31.248 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-30T18:35:31.259 INFO:teuthology.orchestra.console.smithi141:Power on 2023-08-30T18:35:31.260 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-30T18:35:31.268 DEBUG:teuthology.orchestra.console.smithi123:power on output: Chassis Power Control: Up/On 2023-08-30T18:35:31.269 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:35:31.281 DEBUG:teuthology.orchestra.console.smithi141:power on output: Chassis Power Control: Up/On 2023-08-30T18:35:31.282 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:35:31.289 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-08-30T18:35:31.305 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-08-30T18:35:35.294 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:35:35.306 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:35:35.403 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-08-30T18:35:35.415 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-08-30T18:35:39.407 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:35:39.416 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:35:39.516 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-08-30T18:35:39.617 INFO:teuthology.orchestra.console.smithi123:Power on completed 2023-08-30T18:35:39.625 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is on 2023-08-30T18:35:39.719 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish 2023-08-30T18:35:39.719 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:35:39.726 INFO:teuthology.orchestra.console.smithi141:Power on completed 2023-08-30T18:35:39.827 INFO:teuthology.provision.fog.smithi141:Waiting for deploy to finish 2023-08-30T18:35:39.828 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:35:54.807 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:35:54.908 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:36:09.914 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:36:09.983 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:36:25.022 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:36:25.062 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:36:40.127 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:36:40.144 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:36:55.240 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:36:55.242 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:37:10.378 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:37:10.379 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:37:25.510 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:37:25.511 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:37:40.622 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:37:40.623 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:37:55.714 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:37:55.722 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:38:10.822 DEBUG:teuthology.provision.fog.smithi141:FOG request: GET /task/active None 2023-08-30T18:38:10.823 DEBUG:teuthology.provision.fog.smithi123:FOG request: GET /task/active None 2023-08-30T18:38:10.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:38:10.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:38:13.984 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:38:19.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:38:23.070 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:38:29.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:38:32.154 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:38:38.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:38:41.210 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:38:47.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:38:50.266 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:38:56.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:38:59.322 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:39:05.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:39:08.378 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:39:11.084 ERROR:teuthology.orchestra.connection:Error authenticating with smithi123.front.sepia.ceph.com: Authentication timeout. 2023-08-30T18:39:11.086 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-30T18:39:11.087 DEBUG:teuthology.orchestra.run.smithi123:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-30T18:39:14.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:39:32.762 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:39:38.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:39:41.822 WARNING:teuthology.provision.fog.smithi141:[Errno None] Unable to connect to port 22 on 172.21.15.141 2023-08-30T18:39:47.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:39:49.546 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-30T18:39:49.547 DEBUG:teuthology.orchestra.run.smithi141:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-30T18:40:00.410 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-08-30T18:40:00.412 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi123 2023-08-30T18:40:09.413 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-08-30T18:40:09.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:40:09.641 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-08-30T18:40:09.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-08-30T18:40:09.958 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-30T18:40:09.958 DEBUG:teuthology.orchestra.run.smithi123:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-30T18:40:14.730 INFO:teuthology.provision.fog.smithi141:Node is ready 2023-08-30T18:40:14.730 DEBUG:teuthology.orchestra.run.smithi141:> hostname 2023-08-30T18:40:14.749 INFO:teuthology.orchestra.run.smithi141.stdout:smithi141.front.sepia.ceph.com 2023-08-30T18:40:14.749 DEBUG:teuthology.orchestra.run.smithi141:> grep smithi141.front.sepia.ceph.com /etc/hosts 2023-08-30T18:40:14.806 INFO:teuthology.orchestra.run.smithi141.stdout:172.21.15.141 smithi141.front.sepia.ceph.com smithi141 2023-08-30T18:40:14.807 DEBUG:teuthology.orchestra.run.smithi141:> sudo hostname smithi141 2023-08-30T18:40:14.921 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/smithi141.front.sepia.ceph.com/smithi141/g' /etc/hosts 2023-08-30T18:40:14.952 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/172.21.15.141/172.21.15.141/g' /etc/hosts 2023-08-30T18:40:14.974 INFO:teuthology.provision.fog.smithi123:Node is ready 2023-08-30T18:40:14.975 DEBUG:teuthology.orchestra.run.smithi123:> hostname 2023-08-30T18:40:14.993 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com 2023-08-30T18:40:14.993 DEBUG:teuthology.orchestra.run.smithi123:> grep smithi123.front.sepia.ceph.com /etc/hosts 2023-08-30T18:40:15.023 DEBUG:teuthology.orchestra.run.smithi141:> sudo sed -i -e 's/smithi141.front.sepia.ceph.com/smithi141/g' /etc/hostname 2023-08-30T18:40:15.050 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123 2023-08-30T18:40:15.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname smithi123 2023-08-30T18:40:15.092 DEBUG:teuthology.orchestra.run.smithi141:> sudo hostname smithi141 2023-08-30T18:40:15.161 DEBUG:teuthology.orchestra.run.smithi141:> cat /etc/os-release 2023-08-30T18:40:15.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/smithi123.front.sepia.ceph.com/smithi123/g' /etc/hosts 2023-08-30T18:40:15.215 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="CentOS Stream" 2023-08-30T18:40:15.216 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="9" 2023-08-30T18:40:15.216 INFO:teuthology.orchestra.run.smithi141.stdout:ID="centos" 2023-08-30T18:40:15.216 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="rhel fedora" 2023-08-30T18:40:15.216 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="9" 2023-08-30T18:40:15.217 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el9" 2023-08-30T18:40:15.217 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-30T18:40:15.217 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31" 2023-08-30T18:40:15.217 INFO:teuthology.orchestra.run.smithi141.stdout:LOGO="fedora-logo-icon" 2023-08-30T18:40:15.217 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-30T18:40:15.217 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://centos.org/" 2023-08-30T18:40:15.218 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-30T18:40:15.218 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-30T18:40:15.218 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-30T18:40:15.219 INFO:teuthology.provision.fog.smithi141:Deploy complete! 2023-08-30T18:40:15.220 DEBUG:teuthology.parallel:result is None 2023-08-30T18:40:15.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/172.21.15.123/172.21.15.123/g' /etc/hosts 2023-08-30T18:40:15.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/smithi123.front.sepia.ceph.com/smithi123/g' /etc/hostname 2023-08-30T18:40:15.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname smithi123 2023-08-30T18:40:15.439 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-08-30T18:40:15.495 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2023-08-30T18:40:15.495 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2023-08-30T18:40:15.496 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2023-08-30T18:40:15.496 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2023-08-30T18:40:15.496 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2023-08-30T18:40:15.496 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2023-08-30T18:40:15.497 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-30T18:40:15.497 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-08-30T18:40:15.497 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2023-08-30T18:40:15.497 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-30T18:40:15.497 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2023-08-30T18:40:15.498 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-30T18:40:15.498 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-30T18:40:15.498 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-30T18:40:15.499 INFO:teuthology.provision.fog.smithi123:Deploy complete! 2023-08-30T18:40:15.499 DEBUG:teuthology.parallel:result is None 2023-08-30T18:40:15.656 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com 2023-08-30T18:40:15.657 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-30T18:40:15.686 INFO:teuthology.lock.ops:Checking smithi141.front.sepia.ceph.com 2023-08-30T18:40:15.687 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-30T18:40:15.712 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server 2023-08-30T18:40:15.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-08-30T18:40:15.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:40:15.840 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-08-30T18:40:15.913 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-08-30T18:40:15.914 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-08-30T18:40:15.968 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-08-30T18:40:15.969 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-08-30T18:40:16.023 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2023-08-30T18:40:16.023 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2023-08-30T18:40:16.024 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2023-08-30T18:40:16.024 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2023-08-30T18:40:16.024 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2023-08-30T18:40:16.024 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2023-08-30T18:40:16.024 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-30T18:40:16.024 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-08-30T18:40:16.025 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2023-08-30T18:40:16.025 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-30T18:40:16.025 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2023-08-30T18:40:16.025 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-30T18:40:16.025 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-30T18:40:16.026 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-30T18:40:16.026 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-08-30T18:40:16.053 INFO:teuthology.lock.ops:Updating [smithi141.front.sepia.ceph.com]: set os type and version on server 2023-08-30T18:40:16.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-08-30T18:40:16.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi141.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-30T18:40:16.137 DEBUG:teuthology.orchestra.run.smithi141:> true 2023-08-30T18:40:16.165 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi141.front.sepia.ceph.com' 2023-08-30T18:40:16.166 DEBUG:teuthology.orchestra.run.smithi141:> uname -m 2023-08-30T18:40:16.221 INFO:teuthology.orchestra.run.smithi141.stdout:x86_64 2023-08-30T18:40:16.222 DEBUG:teuthology.orchestra.run.smithi141:> cat /etc/os-release 2023-08-30T18:40:16.277 INFO:teuthology.orchestra.run.smithi141.stdout:NAME="CentOS Stream" 2023-08-30T18:40:16.277 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION="9" 2023-08-30T18:40:16.277 INFO:teuthology.orchestra.run.smithi141.stdout:ID="centos" 2023-08-30T18:40:16.278 INFO:teuthology.orchestra.run.smithi141.stdout:ID_LIKE="rhel fedora" 2023-08-30T18:40:16.278 INFO:teuthology.orchestra.run.smithi141.stdout:VERSION_ID="9" 2023-08-30T18:40:16.278 INFO:teuthology.orchestra.run.smithi141.stdout:PLATFORM_ID="platform:el9" 2023-08-30T18:40:16.278 INFO:teuthology.orchestra.run.smithi141.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-30T18:40:16.279 INFO:teuthology.orchestra.run.smithi141.stdout:ANSI_COLOR="0;31" 2023-08-30T18:40:16.279 INFO:teuthology.orchestra.run.smithi141.stdout:LOGO="fedora-logo-icon" 2023-08-30T18:40:16.279 INFO:teuthology.orchestra.run.smithi141.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-30T18:40:16.279 INFO:teuthology.orchestra.run.smithi141.stdout:HOME_URL="https://centos.org/" 2023-08-30T18:40:16.280 INFO:teuthology.orchestra.run.smithi141.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-30T18:40:16.280 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-30T18:40:16.280 INFO:teuthology.orchestra.run.smithi141.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-30T18:40:16.281 INFO:teuthology.lock.ops:Updating smithi141.front.sepia.ceph.com on lock server 2023-08-30T18:40:16.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-30T18:40:16.397 INFO:teuthology.dispatcher.supervisor:Running job 7384613 2023-08-30T18:40:16.397 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f9a8556d5e989d27b6626a0b75ef595991f83d20/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-08-30_18:17:27-rados-wip-yuri6-testing-2023-08-28-1308-distro-default-smithi/7384613 --name yuriw-2023-08-30_18:17:27-rados-wip-yuri6-testing-2023-08-28-1308-distro-default-smithi --description rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_latest} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} -- /home/teuthworker/archive/yuriw-2023-08-30_18:17:27-rados-wip-yuri6-testing-2023-08-28-1308-distro-default-smithi/7384613/orig.config.yaml 2023-08-30T18:40:16.401 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-08-30_18:17:27-rados-wip-yuri6-testing-2023-08-28-1308-distro-default-smithi/7384613 2023-08-30T18:40:16.401 INFO:teuthology.dispatcher.supervisor:Job PID: 2069362 2023-08-30T18:40:16.402 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-30T18:42:16.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-30T18:42:16.574 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-30T18:42:16.686 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-30T18:42:16.687 INFO:teuthology.nuke:Checking targets against current locks 2023-08-30T18:42:16.744 DEBUG:teuthology.nuke:shortname: smithi123 2023-08-30T18:42:16.744 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-30T18:42:16.761 DEBUG:teuthology.nuke:shortname: smithi141 2023-08-30T18:42:16.761 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-30T18:42:16.771 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-30_18:17:27-rados-wip-yuri6-testing-2023-08-28-1308-distro-default-smithi/7384613', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-30 18:35:29.485190', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPo4fLLCvdKJ3+u60XbEVoQ+Az3Ou40YUULwcYFrj6XtfMi6ccgTj7Kqa+SVwdhjew0DSvI8ZishaE9UCIaV5k='} 2023-08-30T18:42:16.783 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi141.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-30_18:17:27-rados-wip-yuri6-testing-2023-08-28-1308-distro-default-smithi/7384613', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-30 18:35:29.487052', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAO1aEnCtNsoKTMRwK9hgsvfxGrzRGfVYcD2eXHepmFsUJM4gyhRi9n2qXFIUGbebkJzjY7AjienibLquWsJox8='} 2023-08-30T18:42:16.814 INFO:teuthology.orchestra.console.smithi123:Power off 2023-08-30T18:42:16.815 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-30T18:42:16.829 INFO:teuthology.orchestra.console.smithi141:Power off 2023-08-30T18:42:16.829 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-30T18:42:16.837 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-08-30T18:42:16.839 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:42:16.849 DEBUG:teuthology.orchestra.console.smithi141:power off output: Chassis Power Control: Down/Off 2023-08-30T18:42:16.850 DEBUG:teuthology.orchestra.console.smithi141:pexpect command: ipmitool -H smithi141.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-30T18:42:16.858 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-08-30T18:42:16.869 DEBUG:teuthology.orchestra.console.smithi141:check power output: Chassis Power is off 2023-08-30T18:42:16.960 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-08-30T18:42:16.971 INFO:teuthology.orchestra.console.smithi141:Power off completed 2023-08-30T18:42:17.105 INFO:teuthology.lock.ops:unlocked: smithi123.front.sepia.ceph.com 2023-08-30T18:42:17.125 INFO:teuthology.lock.ops:unlocked: smithi141.front.sepia.ceph.com