2023-10-10T10:01:23.572 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-10T10:01:23.573 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:01:23.715 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faff9984670>, signals=[15]) 2023-10-10T10:01:23.717 INFO:teuthology.lock.ops:Start node 'smithi032.front.sepia.ceph.com' reimaging 2023-10-10T10:01:23.717 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: reset os type and version on server 2023-10-10T10:01:23.717 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-10-10T10:01:23.739 INFO:teuthology.lock.ops:Start node 'smithi089.front.sepia.ceph.com' reimaging 2023-10-10T10:01:23.739 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: reset os type and version on server 2023-10-10T10:01:23.740 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-10-10T10:01:23.743 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /host {"name": "smithi032"} 2023-10-10T10:01:23.763 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /host {"name": "smithi089"} 2023-10-10T10:01:23.807 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-10-10T10:01:23.839 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-10-10T10:01:23.861 DEBUG:teuthology.provision.fog.smithi032:Requesting image smithi_rhel_8.4 (ID 7) 2023-10-10T10:01:23.861 DEBUG:teuthology.provision.fog.smithi032:FOG request: PUT /host/137 {"imageID": 7} 2023-10-10T10:01:23.876 DEBUG:teuthology.provision.fog.smithi089:Requesting image smithi_rhel_8.4 (ID 7) 2023-10-10T10:01:23.876 DEBUG:teuthology.provision.fog.smithi089:FOG request: PUT /host/194 {"imageID": 7} 2023-10-10T10:01:23.912 INFO:teuthology.provision.fog.smithi032:Scheduling deploy of rhel 8.4 2023-10-10T10:01:23.912 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:01:23.915 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of rhel 8.4 2023-10-10T10:01:23.915 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:01:23.969 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /tasktype {"name": "deploy"} 2023-10-10T10:01:23.971 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /tasktype {"name": "deploy"} 2023-10-10T10:01:24.005 DEBUG:teuthology.provision.fog.smithi089:FOG request: POST /host/194/task {"taskTypeID": 1} 2023-10-10T10:01:24.013 DEBUG:teuthology.provision.fog.smithi032:FOG request: POST /host/137/task {"taskTypeID": 1} 2023-10-10T10:01:24.052 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:01:24.068 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:01:24.147 INFO:teuthology.orchestra.console.smithi089:Power off 2023-10-10T10:01:24.148 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-10T10:01:24.161 INFO:teuthology.orchestra.console.smithi032:Power off 2023-10-10T10:01:24.162 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-10T10:01:24.170 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2023-10-10T10:01:24.171 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:01:24.181 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-10-10T10:01:24.182 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:01:24.190 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-10-10T10:01:24.202 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-10-10T10:01:24.292 INFO:teuthology.orchestra.console.smithi089:Power off completed 2023-10-10T10:01:24.303 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-10-10T10:01:24.394 INFO:teuthology.orchestra.console.smithi089:Power on 2023-10-10T10:01:24.394 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-10T10:01:24.404 INFO:teuthology.orchestra.console.smithi032:Power on 2023-10-10T10:01:24.405 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-10-10T10:01:24.415 DEBUG:teuthology.orchestra.console.smithi089:power on output: Chassis Power Control: Up/On 2023-10-10T10:01:24.416 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:01:24.428 DEBUG:teuthology.orchestra.console.smithi032:power on output: Chassis Power Control: Up/On 2023-10-10T10:01:24.429 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:01:24.437 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-10-10T10:01:24.455 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-10-10T10:01:28.442 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:01:28.456 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:01:28.552 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-10-10T10:01:28.565 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-10-10T10:01:28.666 INFO:teuthology.orchestra.console.smithi032:Power on completed 2023-10-10T10:01:28.766 INFO:teuthology.provision.fog.smithi032:Waiting for deploy to finish 2023-10-10T10:01:28.767 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:01:32.554 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:01:32.668 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2023-10-10T10:01:32.768 INFO:teuthology.orchestra.console.smithi089:Power on completed 2023-10-10T10:01:32.869 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish 2023-10-10T10:01:32.870 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:01:43.838 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:01:47.938 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:01:58.914 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:02:03.003 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:02:13.990 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:02:18.066 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:02:29.071 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:02:33.142 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:02:44.166 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:02:48.218 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:02:59.246 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:03:03.298 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:03:14.318 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:03:18.386 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:03:29.410 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:03:33.471 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:03:44.494 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:03:48.558 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:03:59.598 DEBUG:teuthology.provision.fog.smithi032:FOG request: GET /task/active None 2023-10-10T10:03:59.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:04:03.651 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:04:18.174 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-10-10T10:04:18.733 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-10-10T10:04:18.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:04:24.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:04:27.258 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-10-10T10:04:33.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:04:36.314 WARNING:teuthology.provision.fog.smithi032:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-10-10T10:04:37.370 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-10T10:04:42.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:04:43.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:04:46.426 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-10T10:04:52.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:04:55.482 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-10T10:05:01.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:05:04.574 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-10-10T10:05:10.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:05:42.352 WARNING:teuthology.provision.fog.smithi032:timed out 2023-10-10T10:05:48.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:05:48.642 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-10T10:05:48.643 DEBUG:teuthology.orchestra.run.smithi032:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-10T10:05:48.924 INFO:teuthology.provision.fog.smithi032:Node is ready 2023-10-10T10:05:48.925 DEBUG:teuthology.orchestra.run.smithi032:> hostname 2023-10-10T10:05:48.982 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032.front.sepia.ceph.com 2023-10-10T10:05:48.982 DEBUG:teuthology.orchestra.run.smithi032:> grep smithi032.front.sepia.ceph.com /etc/hosts 2023-10-10T10:05:49.039 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32 smithi032.front.sepia.ceph.com smithi032 2023-10-10T10:05:49.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2023-10-10T10:05:49.166 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi032.front.sepia.ceph.com/smithi032/g' /etc/hosts 2023-10-10T10:05:49.195 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/172.21.15.32/172.21.15.32/g' /etc/hosts 2023-10-10T10:05:49.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi032.front.sepia.ceph.com/smithi032/g' /etc/hostname 2023-10-10T10:05:49.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2023-10-10T10:05:49.393 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-10-10T10:05:49.448 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-10-10T10:05:49.448 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.4 (Ootpa)" 2023-10-10T10:05:49.449 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-10-10T10:05:49.449 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-10-10T10:05:49.449 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.4" 2023-10-10T10:05:49.449 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-10-10T10:05:49.450 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-10T10:05:49.450 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-10-10T10:05:49.450 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-10T10:05:49.450 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-10-10T10:05:49.450 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-10T10:05:49.451 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-10T10:05:49.451 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-10T10:05:49.451 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-10T10:05:49.451 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-10T10:05:49.451 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-10T10:05:49.451 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-10T10:05:49.452 INFO:teuthology.provision.fog.smithi032:Deploy complete! 2023-10-10T10:05:49.453 DEBUG:teuthology.parallel:result is None 2023-10-10T10:06:10.603 WARNING:teuthology.provision.fog.smithi089:timed out 2023-10-10T10:06:16.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:06:16.974 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-10T10:06:16.974 DEBUG:teuthology.orchestra.run.smithi089:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-10-10T10:06:17.255 INFO:teuthology.provision.fog.smithi089:Node is ready 2023-10-10T10:06:17.256 DEBUG:teuthology.orchestra.run.smithi089:> hostname 2023-10-10T10:06:17.271 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com 2023-10-10T10:06:17.271 DEBUG:teuthology.orchestra.run.smithi089:> grep smithi089.front.sepia.ceph.com /etc/hosts 2023-10-10T10:06:17.326 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089 2023-10-10T10:06:17.326 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089 2023-10-10T10:06:17.431 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hosts 2023-10-10T10:06:17.456 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/172.21.15.89/172.21.15.89/g' /etc/hosts 2023-10-10T10:06:17.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hostname 2023-10-10T10:06:17.588 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089 2023-10-10T10:06:17.651 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-10-10T10:06:17.705 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Red Hat Enterprise Linux" 2023-10-10T10:06:17.706 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8.4 (Ootpa)" 2023-10-10T10:06:17.706 INFO:teuthology.orchestra.run.smithi089.stdout:ID="rhel" 2023-10-10T10:06:17.706 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="fedora" 2023-10-10T10:06:17.706 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8.4" 2023-10-10T10:06:17.706 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2023-10-10T10:06:17.707 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-10T10:06:17.707 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2023-10-10T10:06:17.707 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-10T10:06:17.707 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.redhat.com/" 2023-10-10T10:06:17.707 INFO:teuthology.orchestra.run.smithi089.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-10T10:06:17.708 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-10T10:06:17.708 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T10:06:17.708 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-10T10:06:17.708 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-10T10:06:17.708 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-10T10:06:17.709 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-10T10:06:17.709 INFO:teuthology.provision.fog.smithi089:Deploy complete! 2023-10-10T10:06:17.710 DEBUG:teuthology.parallel:result is None 2023-10-10T10:06:17.854 INFO:teuthology.lock.ops:Checking smithi032.front.sepia.ceph.com 2023-10-10T10:06:17.855 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-10T10:06:17.880 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com 2023-10-10T10:06:17.880 INFO:teuthology.lock.ops:New key found. Updating... 2023-10-10T10:06:17.904 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: set os type and version on server 2023-10-10T10:06:17.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-10T10:06:17.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:06:17.977 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-10-10T10:06:18.102 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-10-10T10:06:18.102 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-10-10T10:06:18.157 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-10-10T10:06:18.157 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-10-10T10:06:18.212 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2023-10-10T10:06:18.212 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.4 (Ootpa)" 2023-10-10T10:06:18.212 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2023-10-10T10:06:18.213 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2023-10-10T10:06:18.213 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.4" 2023-10-10T10:06:18.213 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2023-10-10T10:06:18.213 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-10T10:06:18.214 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2023-10-10T10:06:18.214 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-10T10:06:18.214 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2023-10-10T10:06:18.214 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-10T10:06:18.214 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-10T10:06:18.215 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-10-10T10:06:18.215 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-10T10:06:18.215 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-10T10:06:18.215 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-10T10:06:18.215 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-10T10:06:18.216 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-10-10T10:06:18.237 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server 2023-10-10T10:06:18.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-10T10:06:18.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-10T10:06:18.303 DEBUG:teuthology.orchestra.run.smithi089:> true 2023-10-10T10:06:18.368 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2023-10-10T10:06:18.368 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2023-10-10T10:06:18.423 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2023-10-10T10:06:18.423 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2023-10-10T10:06:18.477 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="Red Hat Enterprise Linux" 2023-10-10T10:06:18.477 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8.4 (Ootpa)" 2023-10-10T10:06:18.477 INFO:teuthology.orchestra.run.smithi089.stdout:ID="rhel" 2023-10-10T10:06:18.478 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="fedora" 2023-10-10T10:06:18.478 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8.4" 2023-10-10T10:06:18.478 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2023-10-10T10:06:18.478 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-10T10:06:18.478 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2023-10-10T10:06:18.479 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-10T10:06:18.479 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://www.redhat.com/" 2023-10-10T10:06:18.479 INFO:teuthology.orchestra.run.smithi089.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-10T10:06:18.479 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-10T10:06:18.479 INFO:teuthology.orchestra.run.smithi089.stdout: 2023-10-10T10:06:18.480 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-10T10:06:18.480 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-10T10:06:18.480 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-10T10:06:18.480 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-10T10:06:18.481 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-10-10T10:06:18.504 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:06:18.603 INFO:teuthology.dispatcher.supervisor:Running job 7419506 2023-10-10T10:06:18.603 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-10_04:31:01-fs-wip-yuri3-testing-2023-10-09-0727-quincy-distro-default-smithi/7419506 --name yuriw-2023-10-10_04:31:01-fs-wip-yuri3-testing-2023-10-09-0727-quincy-distro-default-smithi --description fs/permission/{begin/{0-install 1-ceph 2-logrotate} clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_misc} -- /home/teuthworker/archive/yuriw-2023-10-10_04:31:01-fs-wip-yuri3-testing-2023-10-09-0727-quincy-distro-default-smithi/7419506/orig.config.yaml 2023-10-10T10:06:18.607 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-10-10_04:31:01-fs-wip-yuri3-testing-2023-10-09-0727-quincy-distro-default-smithi/7419506 2023-10-10T10:06:18.608 INFO:teuthology.dispatcher.supervisor:Job PID: 963254 2023-10-10T10:06:18.608 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-10-10T10:08:18.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:10:18.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:12:19.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:14:19.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:16:19.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:18:19.375 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:20:19.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:22:19.699 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:24:19.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:26:20.139 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-10T10:26:20.208 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-10-10T10:26:20.316 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-10-10T10:26:20.317 INFO:teuthology.nuke:Checking targets against current locks 2023-10-10T10:26:20.364 DEBUG:teuthology.nuke:shortname: smithi032 2023-10-10T10:26:20.365 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-10T10:26:20.388 DEBUG:teuthology.nuke:shortname: smithi089 2023-10-10T10:26:20.388 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-10T10:26:20.392 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_04:31:01-fs-wip-yuri3-testing-2023-10-09-0727-quincy-distro-default-smithi/7419506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 10:01:22.731955', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqRFN9aXTM+o4qxmKoaGTjEpgO68Ia5PxubJgR7leYNLaa1qlrDvPCjyZbikK+kL/YG47hXC6h/lTq1+3InH9M='} 2023-10-10T10:26:20.413 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-10_04:31:01-fs-wip-yuri3-testing-2023-10-09-0727-quincy-distro-default-smithi/7419506', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-10 10:01:22.730431', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE6wakkjWWkXZj0UQV/eO0zClIdh/YlZZvyd0SYsfvacJvpzhCZ1mllL8/3e497bM1Pfl/MQ8j8MsT01wF7B/Rs='} 2023-10-10T10:26:20.450 INFO:teuthology.orchestra.console.smithi032:Power off 2023-10-10T10:26:20.451 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-10T10:26:20.471 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-10-10T10:26:20.472 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:26:20.486 INFO:teuthology.orchestra.console.smithi089:Power off 2023-10-10T10:26:20.487 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-10T10:26:20.497 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-10-10T10:26:20.510 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2023-10-10T10:26:20.510 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-10T10:26:20.531 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2023-10-10T10:26:20.598 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-10-10T10:26:20.633 INFO:teuthology.orchestra.console.smithi089:Power off completed 2023-10-10T10:26:20.749 INFO:teuthology.lock.ops:unlocked: smithi032.front.sepia.ceph.com 2023-10-10T10:26:20.782 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com