2023-08-11T11:46:12.017 INFO:root:teuthology version: 0.0.1.dev181+g7fda959 2023-08-11T11:46:12.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T11:46:12.166 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb84e4a2ee0>, signals=[15]) 2023-08-11T11:46:12.168 INFO:teuthology.lock.ops:Start node 'smithi037.front.sepia.ceph.com' reimaging 2023-08-11T11:46:12.168 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: reset os type and version on server 2023-08-11T11:46:12.169 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2023-08-11T11:46:12.193 INFO:teuthology.lock.ops:Start node 'smithi121.front.sepia.ceph.com' reimaging 2023-08-11T11:46:12.194 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: reset os type and version on server 2023-08-11T11:46:12.194 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-08-11T11:46:12.196 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /host {"name": "smithi037"} 2023-08-11T11:46:12.219 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /host {"name": "smithi121"} 2023-08-11T11:46:12.260 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-11T11:46:12.279 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-11T11:46:12.294 DEBUG:teuthology.provision.fog.smithi037:Requesting image smithi_centos_8.stream (ID 3) 2023-08-11T11:46:12.294 DEBUG:teuthology.provision.fog.smithi037:FOG request: PUT /host/142 {"imageID": 3} 2023-08-11T11:46:12.319 DEBUG:teuthology.provision.fog.smithi121:Requesting image smithi_centos_8.stream (ID 3) 2023-08-11T11:46:12.319 DEBUG:teuthology.provision.fog.smithi121:FOG request: PUT /host/226 {"imageID": 3} 2023-08-11T11:46:12.331 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of centos 8.stream 2023-08-11T11:46:12.331 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:46:12.362 INFO:teuthology.provision.fog.smithi121:Scheduling deploy of centos 8.stream 2023-08-11T11:46:12.362 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:46:12.389 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /tasktype {"name": "deploy"} 2023-08-11T11:46:12.408 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /tasktype {"name": "deploy"} 2023-08-11T11:46:12.428 DEBUG:teuthology.provision.fog.smithi037:FOG request: POST /host/142/task {"taskTypeID": 1} 2023-08-11T11:46:12.447 DEBUG:teuthology.provision.fog.smithi121:FOG request: POST /host/226/task {"taskTypeID": 1} 2023-08-11T11:46:12.484 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:46:12.493 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:46:12.589 INFO:teuthology.orchestra.console.smithi037:Power off 2023-08-11T11:46:12.590 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T11:46:12.605 INFO:teuthology.orchestra.console.smithi121:Power off 2023-08-11T11:46:12.606 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T11:46:12.615 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2023-08-11T11:46:12.615 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T11:46:12.626 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off 2023-08-11T11:46:12.627 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T11:46:12.637 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-08-11T11:46:12.652 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2023-08-11T11:46:12.739 INFO:teuthology.orchestra.console.smithi037:Power off completed 2023-08-11T11:46:12.754 INFO:teuthology.orchestra.console.smithi121:Power off completed 2023-08-11T11:46:12.840 INFO:teuthology.orchestra.console.smithi037:Power on 2023-08-11T11:46:12.841 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-11T11:46:12.855 INFO:teuthology.orchestra.console.smithi121:Power on 2023-08-11T11:46:12.856 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-11T11:46:12.866 DEBUG:teuthology.orchestra.console.smithi037:power on output: Chassis Power Control: Up/On 2023-08-11T11:46:12.868 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T11:46:12.878 DEBUG:teuthology.orchestra.console.smithi121:power on output: Chassis Power Control: Up/On 2023-08-11T11:46:12.878 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T11:46:12.889 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-08-11T11:46:12.898 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2023-08-11T11:46:16.892 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T11:46:16.901 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T11:46:17.003 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-08-11T11:46:17.013 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2023-08-11T11:46:21.008 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T11:46:21.017 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T11:46:21.119 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2023-08-11T11:46:21.129 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is on 2023-08-11T11:46:21.221 INFO:teuthology.orchestra.console.smithi037:Power on completed 2023-08-11T11:46:21.230 INFO:teuthology.orchestra.console.smithi121:Power on completed 2023-08-11T11:46:21.322 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish 2023-08-11T11:46:21.323 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:46:21.331 INFO:teuthology.provision.fog.smithi121:Waiting for deploy to finish 2023-08-11T11:46:21.332 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:46:36.396 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:46:36.397 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:46:51.476 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:46:51.478 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:47:06.565 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:47:06.566 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:47:21.642 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:47:21.643 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:47:36.716 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:47:36.717 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:47:51.788 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:47:51.789 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:48:06.871 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:48:06.872 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:48:21.952 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:48:21.953 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:48:37.036 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:48:37.038 DEBUG:teuthology.provision.fog.smithi037:FOG request: GET /task/active None 2023-08-11T11:48:37.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:48:37.240 ERROR:teuthology.orchestra.connection:Error authenticating with smithi037.front.sepia.ceph.com: Authentication failed. 2023-08-11T11:48:37.242 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-11T11:48:37.243 DEBUG:teuthology.orchestra.run.smithi037:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-11T11:48:37.244 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-11T11:48:52.124 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:49:07.214 DEBUG:teuthology.provision.fog.smithi121:FOG request: GET /task/active None 2023-08-11T11:49:07.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:49:10.356 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-08-11T11:49:16.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:49:19.408 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-08-11T11:49:25.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:49:28.464 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-08-11T11:49:34.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:49:37.520 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-08-11T11:49:43.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:49:46.576 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-08-11T11:49:52.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:49:55.632 WARNING:teuthology.provision.fog.smithi121:[Errno None] Unable to connect to port 22 on 172.21.15.121 2023-08-11T11:50:01.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:50:29.232 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-08-11T11:50:29.234 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi037 2023-08-11T11:50:38.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-08-11T11:50:38.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:50:38.461 DEBUG:teuthology.orchestra.run.smithi037:> true 2023-08-11T11:50:38.706 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-08-11T11:50:38.707 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-11T11:50:38.707 DEBUG:teuthology.orchestra.run.smithi037:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-11T11:50:38.726 INFO:teuthology.provision.fog.smithi037:Node is ready 2023-08-11T11:50:38.726 DEBUG:teuthology.orchestra.run.smithi037:> hostname 2023-08-11T11:50:38.787 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com 2023-08-11T11:50:38.787 DEBUG:teuthology.orchestra.run.smithi037:> grep smithi037.front.sepia.ceph.com /etc/hosts 2023-08-11T11:50:38.848 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037 2023-08-11T11:50:38.849 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname smithi037 2023-08-11T11:50:38.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/smithi037.front.sepia.ceph.com/smithi037/g' /etc/hosts 2023-08-11T11:50:38.991 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/172.21.15.37/172.21.15.37/g' /etc/hosts 2023-08-11T11:50:39.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/smithi037.front.sepia.ceph.com/smithi037/g' /etc/hostname 2023-08-11T11:50:39.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname smithi037 2023-08-11T11:50:39.202 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2023-08-11T11:50:39.261 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2023-08-11T11:50:39.262 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8" 2023-08-11T11:50:39.262 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2023-08-11T11:50:39.263 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2023-08-11T11:50:39.263 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8" 2023-08-11T11:50:39.263 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8" 2023-08-11T11:50:39.263 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-11T11:50:39.263 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2023-08-11T11:50:39.264 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-11T11:50:39.264 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2023-08-11T11:50:39.264 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T11:50:39.264 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T11:50:39.265 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T11:50:39.266 INFO:teuthology.provision.fog.smithi037:Deploy complete! 2023-08-11T11:50:39.267 DEBUG:teuthology.parallel:result is None 2023-08-11T11:51:01.652 WARNING:teuthology.provision.fog.smithi121:timed out 2023-08-11T11:51:07.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:51:07.875 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-11T11:51:07.876 DEBUG:teuthology.orchestra.run.smithi121:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-11T11:51:08.150 INFO:teuthology.provision.fog.smithi121:Node is ready 2023-08-11T11:51:08.151 DEBUG:teuthology.orchestra.run.smithi121:> hostname 2023-08-11T11:51:08.210 INFO:teuthology.orchestra.run.smithi121.stdout:smithi121.front.sepia.ceph.com 2023-08-11T11:51:08.211 DEBUG:teuthology.orchestra.run.smithi121:> grep smithi121.front.sepia.ceph.com /etc/hosts 2023-08-11T11:51:08.270 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.15.121 smithi121.front.sepia.ceph.com smithi121 2023-08-11T11:51:08.271 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121 2023-08-11T11:51:08.368 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121.front.sepia.ceph.com/smithi121/g' /etc/hosts 2023-08-11T11:51:08.398 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/172.21.15.121/172.21.15.121/g' /etc/hosts 2023-08-11T11:51:08.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i -e 's/smithi121.front.sepia.ceph.com/smithi121/g' /etc/hostname 2023-08-11T11:51:08.535 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname smithi121 2023-08-11T11:51:08.602 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-08-11T11:51:08.660 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2023-08-11T11:51:08.660 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8" 2023-08-11T11:51:08.660 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2023-08-11T11:51:08.661 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2023-08-11T11:51:08.661 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8" 2023-08-11T11:51:08.661 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2023-08-11T11:51:08.661 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-11T11:51:08.661 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2023-08-11T11:51:08.661 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-11T11:51:08.661 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2023-08-11T11:51:08.662 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T11:51:08.662 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T11:51:08.662 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T11:51:08.662 INFO:teuthology.provision.fog.smithi121:Deploy complete! 2023-08-11T11:51:08.663 DEBUG:teuthology.parallel:result is None 2023-08-11T11:51:08.799 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com 2023-08-11T11:51:08.800 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-11T11:51:08.821 INFO:teuthology.lock.ops:Checking smithi121.front.sepia.ceph.com 2023-08-11T11:51:08.822 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-11T11:51:08.839 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server 2023-08-11T11:51:08.839 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-08-11T11:51:08.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:51:08.908 DEBUG:teuthology.orchestra.run.smithi037:> true 2023-08-11T11:51:09.048 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2023-08-11T11:51:09.048 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2023-08-11T11:51:09.107 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2023-08-11T11:51:09.108 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2023-08-11T11:51:09.167 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2023-08-11T11:51:09.168 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="8" 2023-08-11T11:51:09.168 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2023-08-11T11:51:09.168 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2023-08-11T11:51:09.168 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="8" 2023-08-11T11:51:09.168 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el8" 2023-08-11T11:51:09.168 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-11T11:51:09.169 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2023-08-11T11:51:09.169 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-11T11:51:09.169 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2023-08-11T11:51:09.169 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T11:51:09.169 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T11:51:09.169 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T11:51:09.170 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2023-08-11T11:51:09.188 INFO:teuthology.lock.ops:Updating [smithi121.front.sepia.ceph.com]: set os type and version on server 2023-08-11T11:51:09.188 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-08-11T11:51:09.189 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-11T11:51:09.253 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-08-11T11:51:09.318 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-08-11T11:51:09.318 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2023-08-11T11:51:09.376 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2023-08-11T11:51:09.377 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-08-11T11:51:09.434 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2023-08-11T11:51:09.434 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8" 2023-08-11T11:51:09.434 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2023-08-11T11:51:09.434 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2023-08-11T11:51:09.435 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8" 2023-08-11T11:51:09.435 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2023-08-11T11:51:09.435 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-11T11:51:09.435 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2023-08-11T11:51:09.435 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-11T11:51:09.435 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2023-08-11T11:51:09.435 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-11T11:51:09.435 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-11T11:51:09.436 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-11T11:51:09.436 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-08-11T11:51:09.463 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T11:51:09.569 INFO:teuthology.dispatcher.supervisor:Running job 7365710 2023-08-11T11:51:09.569 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_7fda95956ac10132c9b74016ba832db907df09fa/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365710 --name rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/fs/snaps}} -- /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365710/orig.config.yaml 2023-08-11T11:51:09.572 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365710 2023-08-11T11:51:09.573 INFO:teuthology.dispatcher.supervisor:Job PID: 6572 2023-08-11T11:51:09.574 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-11T11:53:09.674 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T11:55:09.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T11:57:10.005 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T11:59:10.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:01:10.325 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:03:10.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:05:10.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:07:10.825 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:09:10.989 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:11:11.157 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:13:11.343 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:15:11.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:17:11.681 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:19:11.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:21:11.997 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:23:12.153 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:25:12.318 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:27:12.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:29:12.669 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:31:12.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:33:13.001 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:35:13.161 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:37:13.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:39:13.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:41:13.691 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:43:13.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:45:14.009 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:47:14.185 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:49:14.353 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:51:14.537 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:53:14.701 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:55:14.869 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:57:15.033 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T12:59:15.211 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:01:15.379 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:03:15.541 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:05:15.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:07:15.880 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:09:16.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:11:16.218 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:13:16.372 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:15:16.570 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:17:16.727 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:19:16.892 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-11T13:19:16.963 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-11T13:19:17.152 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-11T13:19:17.153 INFO:teuthology.nuke:Checking targets against current locks 2023-08-11T13:19:17.205 DEBUG:teuthology.nuke:shortname: smithi037 2023-08-11T13:19:17.205 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T13:19:17.219 DEBUG:teuthology.nuke:shortname: smithi121 2023-08-11T13:19:17.220 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-11T13:19:17.224 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365710', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 11:46:11.201744', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLxEFslTSjYu9DPYnlyvLjQyRq5mD06PURcbkU9sM64fq04e0j1cMg6qqWbQrwoBjFS2IZkJfV8gtdRhaixSxlA='} 2023-08-11T13:19:17.242 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-10_20:16:46-fs-wip-rishabh-2023Aug1-b4-testing-default-smithi/7365710', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-11 11:46:11.202740', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDgGalVqJaFkby6SMZw8W2xKqu/j71YN8Q83zF32TApk2negYJxeLiex8iRDVbkMnTTpQjJs79nOXp6tZCopbQEtmAiD8zf9KGvacZwMM1pDw4Hfb/xylz96ezTbD13PQKV4NmXwkAbQQpm/GdPwlm6S8JYfBOqmZ9HWCqqatSbXcz7Ku7jxgu0SbQRjQ5cEyQThwUz0Noy2sI8TDgmBPVDqbr5aBNYBVQ9shGmKyf8CoKNaTSeXQ4mYLQSCpDd2pyIoFqLBCOTYWOIrOm3JaUKN5w9SDns/RpREDX8xUAlUcyUPWT5XZjTjZt/CZiZ9oa6IBqMW8daYZV4m8TVTPwm9LnHP775bBWcAYE66pBFXtAH9dpf5hfnSWpEjP9pyem+aWhJWOzCMchGNxh8+B+04HMys2HvDMYM++SPGDEm5vMzjImRehUmeYfdroUm2x8/Bb9JxMzn1EE0Qc1naH2dGJbwq4Gwleg5Z8A6eThMrxwX2gdIygfV8Uj6FYuOCAU='} 2023-08-11T13:19:17.271 INFO:teuthology.orchestra.console.smithi037:Power off 2023-08-11T13:19:17.272 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T13:19:17.296 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2023-08-11T13:19:17.297 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:19:17.311 INFO:teuthology.orchestra.console.smithi121:Power off 2023-08-11T13:19:17.312 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-11T13:19:17.322 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2023-08-11T13:19:17.332 DEBUG:teuthology.orchestra.console.smithi121:power off output: Chassis Power Control: Down/Off 2023-08-11T13:19:17.333 DEBUG:teuthology.orchestra.console.smithi121:pexpect command: ipmitool -H smithi121.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-11T13:19:17.354 DEBUG:teuthology.orchestra.console.smithi121:check power output: Chassis Power is off 2023-08-11T13:19:17.424 INFO:teuthology.orchestra.console.smithi037:Power off completed 2023-08-11T13:19:17.456 INFO:teuthology.orchestra.console.smithi121:Power off completed 2023-08-11T13:19:17.571 INFO:teuthology.lock.ops:unlocked: smithi037.front.sepia.ceph.com 2023-08-11T13:19:17.594 INFO:teuthology.lock.ops:unlocked: smithi121.front.sepia.ceph.com