2023-09-27T21:45:51.999 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-27T21:45:51.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T21:45:52.122 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fede5563670>, signals=[15]) 2023-09-27T21:45:52.122 INFO:teuthology.lock.ops:Start node 'smithi053.front.sepia.ceph.com' reimaging 2023-09-27T21:45:52.122 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: reset os type and version on server 2023-09-27T21:45:52.123 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-09-27T21:45:52.145 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging 2023-09-27T21:45:52.145 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server 2023-09-27T21:45:52.145 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-09-27T21:45:52.147 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /host {"name": "smithi053"} 2023-09-27T21:45:52.170 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /host {"name": "smithi118"} 2023-09-27T21:45:52.219 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-09-27T21:45:52.239 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-09-27T21:45:52.266 DEBUG:teuthology.provision.fog.smithi053:Requesting image smithi_rhel_8.4 (ID 7) 2023-09-27T21:45:52.266 DEBUG:teuthology.provision.fog.smithi053:FOG request: PUT /host/158 {"imageID": 7} 2023-09-27T21:45:52.279 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_rhel_8.4 (ID 7) 2023-09-27T21:45:52.279 DEBUG:teuthology.provision.fog.smithi118:FOG request: PUT /host/223 {"imageID": 7} 2023-09-27T21:45:52.308 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of rhel 8.4 2023-09-27T21:45:52.308 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:45:52.333 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of rhel 8.4 2023-09-27T21:45:52.333 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:45:52.367 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /tasktype {"name": "deploy"} 2023-09-27T21:45:52.382 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /tasktype {"name": "deploy"} 2023-09-27T21:45:52.410 DEBUG:teuthology.provision.fog.smithi053:FOG request: POST /host/158/task {"taskTypeID": 1} 2023-09-27T21:45:52.417 DEBUG:teuthology.provision.fog.smithi118:FOG request: POST /host/223/task {"taskTypeID": 1} 2023-09-27T21:45:52.463 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:45:52.469 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:45:52.546 INFO:teuthology.orchestra.console.smithi053:Power off 2023-09-27T21:45:52.546 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-27T21:45:52.560 INFO:teuthology.orchestra.console.smithi118:Power off 2023-09-27T21:45:52.561 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-27T21:45:52.572 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2023-09-27T21:45:52.572 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:45:52.598 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2023-09-27T21:45:52.598 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:45:52.606 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-09-27T21:45:52.620 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-09-27T21:45:56.608 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:45:56.621 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:45:56.717 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-09-27T21:45:56.730 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-09-27T21:46:00.722 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:46:00.730 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:46:00.830 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-09-27T21:46:00.838 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-27T21:46:00.932 INFO:teuthology.orchestra.console.smithi053:Power off completed 2023-09-27T21:46:00.939 INFO:teuthology.orchestra.console.smithi118:Power off completed 2023-09-27T21:46:01.033 INFO:teuthology.orchestra.console.smithi053:Power on 2023-09-27T21:46:01.033 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-27T21:46:01.041 INFO:teuthology.orchestra.console.smithi118:Power on 2023-09-27T21:46:01.041 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-27T21:46:01.055 DEBUG:teuthology.orchestra.console.smithi053:power on output: Chassis Power Control: Up/On 2023-09-27T21:46:01.055 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:46:01.064 DEBUG:teuthology.orchestra.console.smithi118:power on output: Chassis Power Control: Up/On 2023-09-27T21:46:01.064 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:46:01.077 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-09-27T21:46:01.085 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-27T21:46:05.081 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:46:05.088 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:46:05.190 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-09-27T21:46:05.196 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-27T21:46:09.196 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:46:09.203 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T21:46:09.306 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is on 2023-09-27T21:46:09.312 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2023-09-27T21:46:09.408 INFO:teuthology.orchestra.console.smithi053:Power on completed 2023-09-27T21:46:09.413 INFO:teuthology.orchestra.console.smithi118:Power on completed 2023-09-27T21:46:09.509 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish 2023-09-27T21:46:09.509 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:46:09.514 INFO:teuthology.provision.fog.smithi118:Waiting for deploy to finish 2023-09-27T21:46:09.514 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:46:24.577 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:46:24.578 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:46:39.647 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:46:39.651 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:46:54.723 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:46:54.724 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:47:09.797 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:47:09.799 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:47:24.882 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:47:24.883 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:47:39.957 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:47:39.958 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:47:55.034 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:47:55.035 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:48:10.118 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:48:10.120 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:48:25.211 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:48:25.212 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:48:40.292 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:48:40.293 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None 2023-09-27T21:48:40.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:48:55.374 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:48:58.874 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-09-27T21:49:04.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:49:07.930 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-09-27T21:49:10.437 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-09-27T21:49:10.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:49:10.579 ERROR:teuthology.orchestra.connection:Error authenticating with smithi118.front.sepia.ceph.com: Authentication failed. 2023-09-27T21:49:10.579 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-27T21:49:10.580 DEBUG:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-27T21:49:10.581 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-27T21:49:13.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:49:16.986 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53 2023-09-27T21:49:22.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:50:22.990 WARNING:teuthology.provision.fog.smithi053:timed out 2023-09-27T21:50:28.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:50:29.319 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-27T21:50:29.319 DEBUG:teuthology.orchestra.run.smithi053:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-27T21:50:29.559 INFO:teuthology.provision.fog.smithi053:Node is ready 2023-09-27T21:50:29.559 DEBUG:teuthology.orchestra.run.smithi053:> hostname 2023-09-27T21:50:29.616 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053.front.sepia.ceph.com 2023-09-27T21:50:29.616 DEBUG:teuthology.orchestra.run.smithi053:> grep smithi053.front.sepia.ceph.com /etc/hosts 2023-09-27T21:50:29.672 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53 smithi053.front.sepia.ceph.com smithi053 2023-09-27T21:50:29.673 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053 2023-09-27T21:50:29.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hosts 2023-09-27T21:50:29.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/172.21.15.53/172.21.15.53/g' /etc/hosts 2023-09-27T21:50:29.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hostname 2023-09-27T21:50:29.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053 2023-09-27T21:50:30.005 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-09-27T21:50:30.060 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux" 2023-09-27T21:50:30.060 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.4 (Ootpa)" 2023-09-27T21:50:30.060 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel" 2023-09-27T21:50:30.060 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora" 2023-09-27T21:50:30.060 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.4" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-27T21:50:30.061 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-27T21:50:30.062 INFO:teuthology.provision.fog.smithi053:Deploy complete! 2023-09-27T21:50:30.063 DEBUG:teuthology.parallel:result is None 2023-09-27T21:51:29.402 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-09-27T21:51:29.403 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi118 2023-09-27T21:51:38.404 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-09-27T21:51:38.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:51:38.664 DEBUG:teuthology.orchestra.run.smithi118:> true 2023-09-27T21:51:38.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-09-27T21:51:38.927 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-27T21:51:38.927 DEBUG:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-27T21:51:38.942 INFO:teuthology.provision.fog.smithi118:Node is ready 2023-09-27T21:51:38.942 DEBUG:teuthology.orchestra.run.smithi118:> hostname 2023-09-27T21:51:38.998 INFO:teuthology.orchestra.run.smithi118.stdout:smithi118.front.sepia.ceph.com 2023-09-27T21:51:38.999 DEBUG:teuthology.orchestra.run.smithi118:> grep smithi118.front.sepia.ceph.com /etc/hosts 2023-09-27T21:51:39.053 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.15.118 smithi118.front.sepia.ceph.com smithi118 2023-09-27T21:51:39.053 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2023-09-27T21:51:39.249 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hosts 2023-09-27T21:51:39.274 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/172.21.15.118/172.21.15.118/g' /etc/hosts 2023-09-27T21:51:39.339 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hostname 2023-09-27T21:51:39.405 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2023-09-27T21:51:39.467 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2023-09-27T21:51:39.522 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="Red Hat Enterprise Linux" 2023-09-27T21:51:39.522 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="8.4 (Ootpa)" 2023-09-27T21:51:39.522 INFO:teuthology.orchestra.run.smithi118.stdout:ID="rhel" 2023-09-27T21:51:39.522 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="fedora" 2023-09-27T21:51:39.522 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="8.4" 2023-09-27T21:51:39.522 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el8" 2023-09-27T21:51:39.522 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://www.redhat.com/" 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-27T21:51:39.523 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-27T21:51:39.524 INFO:teuthology.provision.fog.smithi118:Deploy complete! 2023-09-27T21:51:39.524 DEBUG:teuthology.parallel:result is None 2023-09-27T21:51:39.667 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com 2023-09-27T21:51:39.667 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-27T21:51:39.689 INFO:teuthology.lock.ops:Checking smithi118.front.sepia.ceph.com 2023-09-27T21:51:39.690 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-27T21:51:39.707 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server 2023-09-27T21:51:39.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-09-27T21:51:39.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:51:39.774 DEBUG:teuthology.orchestra.run.smithi053:> true 2023-09-27T21:51:39.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2023-09-27T21:51:39.844 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2023-09-27T21:51:39.901 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2023-09-27T21:51:39.901 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2023-09-27T21:51:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux" 2023-09-27T21:51:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.4 (Ootpa)" 2023-09-27T21:51:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel" 2023-09-27T21:51:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora" 2023-09-27T21:51:39.959 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.4" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-27T21:51:39.960 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-27T21:51:39.961 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-27T21:51:39.961 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-27T21:51:39.961 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2023-09-27T21:51:39.981 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: set os type and version on server 2023-09-27T21:51:39.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-09-27T21:51:39.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-27T21:51:40.049 DEBUG:teuthology.orchestra.run.smithi118:> true 2023-09-27T21:51:40.112 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2023-09-27T21:51:40.112 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2023-09-27T21:51:40.165 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2023-09-27T21:51:40.166 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2023-09-27T21:51:40.218 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="Red Hat Enterprise Linux" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="8.4 (Ootpa)" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:ID="rhel" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="fedora" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="8.4" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el8" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://www.redhat.com/" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-27T21:51:40.219 INFO:teuthology.orchestra.run.smithi118.stdout: 2023-09-27T21:51:40.220 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-27T21:51:40.220 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-27T21:51:40.220 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-27T21:51:40.220 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-27T21:51:40.220 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-09-27T21:51:40.256 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T21:51:40.351 INFO:teuthology.dispatcher.supervisor:Running job 7403426 2023-09-27T21:51:40.352 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-09-27_19:24:38-rgw-wip-yuri4-testing-2023-09-27-0917-quincy-distro-default-smithi/7403426 --name yuriw-2023-09-27_19:24:38-rgw-wip-yuri4-testing-2023-09-27-0917-quincy-distro-default-smithi --description rgw/multisite/{clusters frontend/beast ignore-pg-availability notify omap_limits overrides realms/two-zonegroup supported-random-distro$/{rhel_8} tasks/test_multi} -- /home/teuthworker/archive/yuriw-2023-09-27_19:24:38-rgw-wip-yuri4-testing-2023-09-27-0917-quincy-distro-default-smithi/7403426/orig.config.yaml 2023-09-27T21:51:40.354 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-09-27_19:24:38-rgw-wip-yuri4-testing-2023-09-27-0917-quincy-distro-default-smithi/7403426 2023-09-27T21:51:40.355 INFO:teuthology.dispatcher.supervisor:Job PID: 3445960 2023-09-27T21:51:40.355 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-27T21:53:40.443 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T21:55:40.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T21:57:40.779 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T21:59:40.943 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:01:41.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:03:41.234 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:05:41.386 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:07:41.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:09:41.649 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:11:41.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:13:41.889 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:15:41.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:17:42.101 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:19:42.209 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:21:42.367 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:23:42.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:25:42.698 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:27:42.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:29:42.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:31:43.059 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:33:43.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:35:43.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:37:43.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:39:43.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:41:43.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:43:43.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:45:44.003 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:47:44.175 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:49:44.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:51:44.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:53:44.675 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:55:44.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:57:44.940 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T22:59:45.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:01:45.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:03:45.430 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:05:45.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:07:45.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:09:45.782 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:11:45.916 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:13:46.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:15:46.243 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:17:46.411 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:19:46.579 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:21:46.744 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:23:46.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:25:47.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-27T23:25:47.177 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-09-27T23:25:47.350 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-27T23:25:47.351 INFO:teuthology.nuke:Checking targets against current locks 2023-09-27T23:25:47.401 DEBUG:teuthology.nuke:shortname: smithi053 2023-09-27T23:25:47.402 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-27T23:25:47.422 DEBUG:teuthology.nuke:shortname: smithi118 2023-09-27T23:25:47.423 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-27T23:25:47.426 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:24:38-rgw-wip-yuri4-testing-2023-09-27-0917-quincy-distro-default-smithi/7403426', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-27 21:45:51.278279', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE0Ue4mq4Kr/zMax0xsIIAw2LrUGZEaX76os7MEUNpLlG2mI9C/NwcQm+fbe5HYSHxvzYhO+V88NSj5R/WJ/BwM='} 2023-09-27T23:25:47.443 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-27_19:24:38-rgw-wip-yuri4-testing-2023-09-27-0917-quincy-distro-default-smithi/7403426', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-27 21:45:51.276889', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBcWlirK+7o/XzrRG09tjoaszo0Zf1KDoQbJOL29rrqmij3RKepCKNIrEI8nnamxHxmOtkOvLaf5AbVSO9Zhp1k='} 2023-09-27T23:25:47.472 INFO:teuthology.orchestra.console.smithi053:Power off 2023-09-27T23:25:47.473 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-27T23:25:47.493 DEBUG:teuthology.orchestra.console.smithi053:power off output: Chassis Power Control: Down/Off 2023-09-27T23:25:47.493 DEBUG:teuthology.orchestra.console.smithi053:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:25:47.505 INFO:teuthology.orchestra.console.smithi118:Power off 2023-09-27T23:25:47.506 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-27T23:25:47.512 DEBUG:teuthology.orchestra.console.smithi053:check power output: Chassis Power is off 2023-09-27T23:25:47.526 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2023-09-27T23:25:47.526 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-27T23:25:47.546 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2023-09-27T23:25:47.613 INFO:teuthology.orchestra.console.smithi053:Power off completed 2023-09-27T23:25:47.648 INFO:teuthology.orchestra.console.smithi118:Power off completed 2023-09-27T23:25:47.761 INFO:teuthology.lock.ops:unlocked: smithi053.front.sepia.ceph.com 2023-09-27T23:25:47.792 INFO:teuthology.lock.ops:unlocked: smithi118.front.sepia.ceph.com