2023-09-29T15:45:47.005 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-29T15:45:47.006 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T15:45:47.148 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fad41a8f670>, signals=[15]) 2023-09-29T15:45:47.148 INFO:teuthology.lock.ops:Start node 'smithi006.front.sepia.ceph.com' reimaging 2023-09-29T15:45:47.149 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: reset os type and version on server 2023-09-29T15:45:47.149 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-09-29T15:45:47.167 INFO:teuthology.lock.ops:Start node 'smithi148.front.sepia.ceph.com' reimaging 2023-09-29T15:45:47.167 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: reset os type and version on server 2023-09-29T15:45:47.168 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-09-29T15:45:47.169 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /host {"name": "smithi006"} 2023-09-29T15:45:47.192 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /host {"name": "smithi148"} 2023-09-29T15:45:47.252 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-09-29T15:45:47.257 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-09-29T15:45:47.295 DEBUG:teuthology.provision.fog.smithi006:Requesting image smithi_rhel_8.4 (ID 7) 2023-09-29T15:45:47.296 DEBUG:teuthology.provision.fog.smithi006:FOG request: PUT /host/112 {"imageID": 7} 2023-09-29T15:45:47.314 DEBUG:teuthology.provision.fog.smithi148:Requesting image smithi_rhel_8.4 (ID 7) 2023-09-29T15:45:47.314 DEBUG:teuthology.provision.fog.smithi148:FOG request: PUT /host/253 {"imageID": 7} 2023-09-29T15:45:47.347 INFO:teuthology.provision.fog.smithi006:Scheduling deploy of rhel 8.4 2023-09-29T15:45:47.347 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:45:47.364 INFO:teuthology.provision.fog.smithi148:Scheduling deploy of rhel 8.4 2023-09-29T15:45:47.364 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:45:47.401 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /tasktype {"name": "deploy"} 2023-09-29T15:45:47.415 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /tasktype {"name": "deploy"} 2023-09-29T15:45:47.445 DEBUG:teuthology.provision.fog.smithi006:FOG request: POST /host/112/task {"taskTypeID": 1} 2023-09-29T15:45:47.449 DEBUG:teuthology.provision.fog.smithi148:FOG request: POST /host/253/task {"taskTypeID": 1} 2023-09-29T15:45:47.509 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:45:47.522 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:45:47.595 INFO:teuthology.orchestra.console.smithi148:Power off 2023-09-29T15:45:47.595 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T15:45:47.608 INFO:teuthology.orchestra.console.smithi006:Power off 2023-09-29T15:45:47.609 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T15:45:47.617 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-09-29T15:45:47.618 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:45:47.631 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2023-09-29T15:45:47.631 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:45:47.640 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-09-29T15:45:47.654 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-09-29T15:45:51.644 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:45:51.655 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:45:51.753 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-09-29T15:45:51.763 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-09-29T15:45:55.758 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:45:55.813 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:45:55.867 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-09-29T15:45:55.927 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-09-29T15:45:59.870 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:45:59.956 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:46:00.059 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-09-29T15:46:00.065 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-09-29T15:46:00.160 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-09-29T15:46:00.166 INFO:teuthology.orchestra.console.smithi006:Power off completed 2023-09-29T15:46:00.261 INFO:teuthology.orchestra.console.smithi148:Power on 2023-09-29T15:46:00.261 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-29T15:46:00.269 INFO:teuthology.orchestra.console.smithi006:Power on 2023-09-29T15:46:00.270 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-29T15:46:00.283 DEBUG:teuthology.orchestra.console.smithi148:power on output: Chassis Power Control: Up/On 2023-09-29T15:46:00.283 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:46:00.293 DEBUG:teuthology.orchestra.console.smithi006:power on output: Chassis Power Control: Up/On 2023-09-29T15:46:00.294 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:46:00.309 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-09-29T15:46:00.313 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-09-29T15:46:04.314 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:46:04.321 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:46:04.424 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-09-29T15:46:04.431 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-09-29T15:46:04.532 INFO:teuthology.orchestra.console.smithi006:Power on completed 2023-09-29T15:46:04.633 INFO:teuthology.provision.fog.smithi006:Waiting for deploy to finish 2023-09-29T15:46:04.633 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:46:08.426 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T15:46:08.535 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2023-09-29T15:46:08.636 INFO:teuthology.orchestra.console.smithi148:Power on completed 2023-09-29T15:46:08.736 INFO:teuthology.provision.fog.smithi148:Waiting for deploy to finish 2023-09-29T15:46:08.737 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:46:19.706 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:46:23.804 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:46:34.790 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:46:38.877 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:46:49.874 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:46:53.946 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:47:04.950 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:47:09.026 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:47:20.022 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:47:24.104 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:47:35.102 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:47:39.174 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:47:50.172 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:47:54.245 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:48:05.367 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:48:09.306 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:48:20.557 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:48:24.370 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:48:35.622 DEBUG:teuthology.provision.fog.smithi006:FOG request: GET /task/active None 2023-09-29T15:48:35.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:48:36.165 ERROR:teuthology.orchestra.connection:Error authenticating with smithi006.front.sepia.ceph.com: Authentication failed. 2023-09-29T15:48:36.166 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-29T15:48:36.166 DEBUG:teuthology.orchestra.run.smithi006:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-29T15:48:36.167 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-29T15:48:37.172 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-29T15:48:38.176 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-29T15:48:39.430 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:48:54.485 DEBUG:teuthology.provision.fog.smithi148:FOG request: GET /task/active None 2023-09-29T15:48:54.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:48:57.594 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-09-29T15:49:03.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:49:06.714 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-09-29T15:49:12.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:49:15.770 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-09-29T15:49:21.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:49:24.826 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-09-29T15:49:30.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:49:33.886 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-09-29T15:49:39.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:49:42.942 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-09-29T15:49:48.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:49:51.994 WARNING:teuthology.provision.fog.smithi148:[Errno None] Unable to connect to port 22 on 172.21.15.148 2023-09-29T15:49:58.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:50:50.235 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-09-29T15:50:50.236 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi006 2023-09-29T15:50:58.256 WARNING:teuthology.provision.fog.smithi148:timed out 2023-09-29T15:50:59.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-09-29T15:50:59.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:50:59.517 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-09-29T15:50:59.785 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-09-29T15:50:59.785 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-29T15:50:59.785 DEBUG:teuthology.orchestra.run.smithi006:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-29T15:50:59.801 INFO:teuthology.provision.fog.smithi006:Node is ready 2023-09-29T15:50:59.801 DEBUG:teuthology.orchestra.run.smithi006:> hostname 2023-09-29T15:50:59.857 INFO:teuthology.orchestra.run.smithi006.stdout:smithi006.front.sepia.ceph.com 2023-09-29T15:50:59.858 DEBUG:teuthology.orchestra.run.smithi006:> grep smithi006.front.sepia.ceph.com /etc/hosts 2023-09-29T15:50:59.915 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.15.6 smithi006.front.sepia.ceph.com smithi006 2023-09-29T15:50:59.915 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname smithi006 2023-09-29T15:51:00.027 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/smithi006.front.sepia.ceph.com/smithi006/g' /etc/hosts 2023-09-29T15:51:00.061 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/172.21.15.6/172.21.15.6/g' /etc/hosts 2023-09-29T15:51:00.131 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i -e 's/smithi006.front.sepia.ceph.com/smithi006/g' /etc/hostname 2023-09-29T15:51:00.204 DEBUG:teuthology.orchestra.run.smithi006:> sudo hostname smithi006 2023-09-29T15:51:00.274 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-09-29T15:51:00.336 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T15:51:00.336 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T15:51:00.337 INFO:teuthology.orchestra.run.smithi006.stdout:ID="rhel" 2023-09-29T15:51:00.337 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="fedora" 2023-09-29T15:51:00.337 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8.4" 2023-09-29T15:51:00.337 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-09-29T15:51:00.337 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T15:51:00.337 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-09-29T15:51:00.337 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T15:51:00.338 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T15:51:00.338 INFO:teuthology.orchestra.run.smithi006.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T15:51:00.339 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T15:51:00.339 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-09-29T15:51:00.339 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T15:51:00.339 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T15:51:00.339 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T15:51:00.339 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T15:51:00.340 INFO:teuthology.provision.fog.smithi006:Deploy complete! 2023-09-29T15:51:00.342 DEBUG:teuthology.parallel:result is None 2023-09-29T15:51:04.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:51:04.553 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-29T15:51:04.554 DEBUG:teuthology.orchestra.run.smithi148:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-29T15:51:04.815 INFO:teuthology.provision.fog.smithi148:Node is ready 2023-09-29T15:51:04.815 DEBUG:teuthology.orchestra.run.smithi148:> hostname 2023-09-29T15:51:04.871 INFO:teuthology.orchestra.run.smithi148.stdout:smithi148.front.sepia.ceph.com 2023-09-29T15:51:04.872 DEBUG:teuthology.orchestra.run.smithi148:> grep smithi148.front.sepia.ceph.com /etc/hosts 2023-09-29T15:51:04.929 INFO:teuthology.orchestra.run.smithi148.stdout:172.21.15.148 smithi148.front.sepia.ceph.com smithi148 2023-09-29T15:51:04.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname smithi148 2023-09-29T15:51:05.047 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/smithi148.front.sepia.ceph.com/smithi148/g' /etc/hosts 2023-09-29T15:51:05.078 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/172.21.15.148/172.21.15.148/g' /etc/hosts 2023-09-29T15:51:05.145 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i -e 's/smithi148.front.sepia.ceph.com/smithi148/g' /etc/hostname 2023-09-29T15:51:05.212 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname smithi148 2023-09-29T15:51:05.278 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-09-29T15:51:05.333 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T15:51:05.333 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T15:51:05.333 INFO:teuthology.orchestra.run.smithi148.stdout:ID="rhel" 2023-09-29T15:51:05.333 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="fedora" 2023-09-29T15:51:05.333 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8.4" 2023-09-29T15:51:05.333 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8" 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T15:51:05.334 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T15:51:05.335 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T15:51:05.335 INFO:teuthology.provision.fog.smithi148:Deploy complete! 2023-09-29T15:51:05.336 DEBUG:teuthology.parallel:result is None 2023-09-29T15:51:05.488 INFO:teuthology.lock.ops:Checking smithi006.front.sepia.ceph.com 2023-09-29T15:51:05.489 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-29T15:51:05.515 INFO:teuthology.lock.ops:Checking smithi148.front.sepia.ceph.com 2023-09-29T15:51:05.515 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-29T15:51:05.533 INFO:teuthology.lock.ops:Updating [smithi006.front.sepia.ceph.com]: set os type and version on server 2023-09-29T15:51:05.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-09-29T15:51:05.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:51:05.598 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-09-29T15:51:05.670 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-09-29T15:51:05.671 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2023-09-29T15:51:05.727 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2023-09-29T15:51:05.727 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-09-29T15:51:05.782 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:ID="rhel" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="fedora" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="8.4" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el8" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T15:51:05.783 INFO:teuthology.orchestra.run.smithi006.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T15:51:05.784 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T15:51:05.784 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-09-29T15:51:05.784 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T15:51:05.784 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T15:51:05.784 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T15:51:05.784 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T15:51:05.784 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-09-29T15:51:05.806 INFO:teuthology.lock.ops:Updating [smithi148.front.sepia.ceph.com]: set os type and version on server 2023-09-29T15:51:05.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-09-29T15:51:05.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T15:51:05.871 DEBUG:teuthology.orchestra.run.smithi148:> true 2023-09-29T15:51:05.942 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2023-09-29T15:51:05.942 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2023-09-29T15:51:05.999 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2023-09-29T15:51:05.999 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2023-09-29T15:51:06.054 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T15:51:06.054 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T15:51:06.054 INFO:teuthology.orchestra.run.smithi148.stdout:ID="rhel" 2023-09-29T15:51:06.054 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="fedora" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="8.4" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el8" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout: 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T15:51:06.055 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T15:51:06.056 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T15:51:06.056 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T15:51:06.056 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2023-09-29T15:51:06.074 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T15:51:06.164 INFO:teuthology.dispatcher.supervisor:Running job 7405418 2023-09-29T15:51:06.164 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-09-28_11:54:11-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7405418 --name yuriw-2023-09-28_11:54:11-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/none objectstore-ec/bluestore-bitmap overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} -- /home/teuthworker/archive/yuriw-2023-09-28_11:54:11-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7405418/orig.config.yaml 2023-09-29T15:51:06.182 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-09-28_11:54:11-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7405418 2023-09-29T15:51:06.182 INFO:teuthology.dispatcher.supervisor:Job PID: 1104147 2023-09-29T15:51:06.182 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-29T15:53:06.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T15:55:06.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T15:57:06.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T15:59:06.787 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T16:01:06.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T16:03:07.122 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T16:05:07.287 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T16:07:07.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T16:07:07.541 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-09-29T16:07:07.691 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-29T16:07:07.691 INFO:teuthology.nuke:Checking targets against current locks 2023-09-29T16:07:07.741 DEBUG:teuthology.nuke:shortname: smithi006 2023-09-29T16:07:07.741 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T16:07:07.765 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-28_11:54:11-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7405418', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 15:45:46.176135', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMwFAH7pCCCaNoHQ9twup5W9mEu/8X1uPxE0YJKlVLuqo7f1HQPRZ9alUd42dwp1JHug/VboJB4ag5UX59iNgGU='} 2023-09-29T16:07:07.769 DEBUG:teuthology.nuke:shortname: smithi148 2023-09-29T16:07:07.769 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T16:07:07.794 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-28_11:54:11-fs-wip-yuri-testing-2023-09-27-0906-quincy-distro-default-smithi/7405418', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 15:45:46.177913', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAwIdssPpUf8eUxnlPpz+VgN1hnt7nRZJCng5KnxhYSFqnyxuYnmkin/J9+L5wJZFeeuAVIUYV51gLo/3CCn44Y='} 2023-09-29T16:07:07.820 INFO:teuthology.orchestra.console.smithi006:Power off 2023-09-29T16:07:07.821 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T16:07:07.842 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2023-09-29T16:07:07.843 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T16:07:07.858 INFO:teuthology.orchestra.console.smithi148:Power off 2023-09-29T16:07:07.859 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T16:07:07.866 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-09-29T16:07:07.878 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2023-09-29T16:07:07.879 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T16:07:07.903 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2023-09-29T16:07:07.968 INFO:teuthology.orchestra.console.smithi006:Power off completed 2023-09-29T16:07:08.004 INFO:teuthology.orchestra.console.smithi148:Power off completed 2023-09-29T16:07:08.120 INFO:teuthology.lock.ops:unlocked: smithi006.front.sepia.ceph.com 2023-09-29T16:07:08.154 INFO:teuthology.lock.ops:unlocked: smithi148.front.sepia.ceph.com