2023-06-22T17:53:55.193 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-22T17:53:55.194 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T17:53:55.195 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T17:53:55.352 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6f6aafc8b0>, signals=[15]) 2023-06-22T17:53:55.354 INFO:teuthology.lock.ops:Start node 'smithi047.front.sepia.ceph.com' reimaging 2023-06-22T17:53:55.354 INFO:teuthology.lock.ops:Updating [smithi047.front.sepia.ceph.com]: reset os type and version on server 2023-06-22T17:53:55.355 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2023-06-22T17:53:55.379 INFO:teuthology.lock.ops:Start node 'smithi202.front.sepia.ceph.com' reimaging 2023-06-22T17:53:55.379 INFO:teuthology.lock.ops:Updating [smithi202.front.sepia.ceph.com]: reset os type and version on server 2023-06-22T17:53:55.379 INFO:teuthology.lock.ops:Updating smithi202.front.sepia.ceph.com on lock server 2023-06-22T17:53:55.380 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /host {"name": "smithi047"} 2023-06-22T17:53:55.403 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /host {"name": "smithi202"} 2023-06-22T17:53:55.458 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-06-22T17:53:55.468 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-06-22T17:53:55.500 DEBUG:teuthology.provision.fog.smithi047:Requesting image smithi_rhel_8.4 (ID 7) 2023-06-22T17:53:55.501 DEBUG:teuthology.provision.fog.smithi047:FOG request: PUT /host/152 {"imageID": 7} 2023-06-22T17:53:55.509 DEBUG:teuthology.provision.fog.smithi202:Requesting image smithi_rhel_8.4 (ID 7) 2023-06-22T17:53:55.509 DEBUG:teuthology.provision.fog.smithi202:FOG request: PUT /host/306 {"imageID": 7} 2023-06-22T17:53:55.540 INFO:teuthology.provision.fog.smithi047:Scheduling deploy of rhel 8.4 2023-06-22T17:53:55.541 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:53:55.544 INFO:teuthology.provision.fog.smithi202:Scheduling deploy of rhel 8.4 2023-06-22T17:53:55.545 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:53:55.595 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /tasktype {"name": "deploy"} 2023-06-22T17:53:55.598 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /tasktype {"name": "deploy"} 2023-06-22T17:53:55.629 DEBUG:teuthology.provision.fog.smithi202:FOG request: POST /host/306/task {"taskTypeID": 1} 2023-06-22T17:53:55.646 DEBUG:teuthology.provision.fog.smithi047:FOG request: POST /host/152/task {"taskTypeID": 1} 2023-06-22T17:53:55.692 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:53:55.700 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:53:55.786 INFO:teuthology.orchestra.console:Power off smithi047 2023-06-22T17:53:55.787 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T17:53:55.804 INFO:teuthology.orchestra.console:Power off smithi202 2023-06-22T17:53:55.804 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T17:53:55.814 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:53:55.825 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:53:55.936 INFO:teuthology.orchestra.console:Power off for smithi047 completed 2023-06-22T17:53:55.947 INFO:teuthology.orchestra.console:Power off for smithi202 completed 2023-06-22T17:53:56.038 INFO:teuthology.orchestra.console:Power on smithi047 2023-06-22T17:53:56.038 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-22T17:53:56.048 INFO:teuthology.orchestra.console:Power on smithi202 2023-06-22T17:53:56.049 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-22T17:53:56.063 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:53:56.071 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:54:00.090 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:54:00.114 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:54:00.324 INFO:teuthology.orchestra.console:Power on for smithi047 completed 2023-06-22T17:54:00.425 INFO:teuthology.provision.fog.smithi047:Waiting for deploy to finish 2023-06-22T17:54:00.426 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:54:04.204 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T17:54:04.415 INFO:teuthology.orchestra.console:Power on for smithi202 completed 2023-06-22T17:54:04.516 INFO:teuthology.provision.fog.smithi202:Waiting for deploy to finish 2023-06-22T17:54:04.516 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:54:15.496 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:54:19.588 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:54:30.556 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:54:34.649 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:54:45.627 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:54:49.706 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:55:00.684 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:55:04.757 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:55:15.745 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:55:19.812 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:55:30.804 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:55:34.872 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:55:45.867 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:55:49.925 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:56:00.928 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:56:04.984 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:56:15.984 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:56:20.040 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:56:31.044 DEBUG:teuthology.provision.fog.smithi047:FOG request: GET /task/active None 2023-06-22T17:56:31.092 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:56:35.094 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:56:49.652 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2023-06-22T17:56:50.146 DEBUG:teuthology.provision.fog.smithi202:FOG request: GET /task/active None 2023-06-22T17:56:50.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:56:55.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:56:58.736 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2023-06-22T17:57:04.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:57:07.792 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2023-06-22T17:57:08.592 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-06-22T17:57:13.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:57:14.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:57:16.852 WARNING:teuthology.provision.fog.smithi047:[Errno None] Unable to connect to port 22 on 172.21.15.47 2023-06-22T17:57:17.652 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-06-22T17:57:22.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:57:23.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:57:26.704 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-06-22T17:57:32.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:57:35.760 WARNING:teuthology.provision.fog.smithi202:[Errno None] Unable to connect to port 22 on 172.21.15.202 2023-06-22T17:57:41.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:58:22.900 WARNING:teuthology.provision.fog.smithi047:timed out 2023-06-22T17:58:28.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:58:29.227 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-22T17:58:29.228 DEBUG:teuthology.orchestra.run.smithi047:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-22T17:58:29.496 INFO:teuthology.provision.fog.smithi047:Node is ready 2023-06-22T17:58:29.496 DEBUG:teuthology.orchestra.run.smithi047:> hostname 2023-06-22T17:58:29.552 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047.front.sepia.ceph.com 2023-06-22T17:58:29.552 DEBUG:teuthology.orchestra.run.smithi047:> grep smithi047.front.sepia.ceph.com /etc/hosts 2023-06-22T17:58:29.608 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47 smithi047.front.sepia.ceph.com smithi047 2023-06-22T17:58:29.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname smithi047 2023-06-22T17:58:29.730 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/smithi047.front.sepia.ceph.com/smithi047/g' /etc/hosts 2023-06-22T17:58:29.757 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/172.21.15.47/172.21.15.47/g' /etc/hosts 2023-06-22T17:58:29.823 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i -e 's/smithi047.front.sepia.ceph.com/smithi047/g' /etc/hostname 2023-06-22T17:58:29.889 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname smithi047 2023-06-22T17:58:29.953 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2023-06-22T17:58:30.007 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T17:58:30.008 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T17:58:30.008 INFO:teuthology.orchestra.run.smithi047.stdout:ID="rhel" 2023-06-22T17:58:30.008 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="fedora" 2023-06-22T17:58:30.008 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="8.4" 2023-06-22T17:58:30.008 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el8" 2023-06-22T17:58:30.008 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T17:58:30.008 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2023-06-22T17:58:30.009 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T17:58:30.009 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T17:58:30.009 INFO:teuthology.orchestra.run.smithi047.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T17:58:30.009 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T17:58:30.009 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:58:30.009 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T17:58:30.009 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T17:58:30.010 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T17:58:30.010 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T17:58:30.010 INFO:teuthology.provision.fog.smithi047:Deploy complete! 2023-06-22T17:58:30.012 DEBUG:teuthology.parallel:result is None 2023-06-22T17:58:41.780 WARNING:teuthology.provision.fog.smithi202:timed out 2023-06-22T17:58:47.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:58:48.063 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-22T17:58:48.063 DEBUG:teuthology.orchestra.run.smithi202:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-22T17:58:48.333 INFO:teuthology.provision.fog.smithi202:Node is ready 2023-06-22T17:58:48.333 DEBUG:teuthology.orchestra.run.smithi202:> hostname 2023-06-22T17:58:48.347 INFO:teuthology.orchestra.run.smithi202.stdout:smithi202.front.sepia.ceph.com 2023-06-22T17:58:48.347 DEBUG:teuthology.orchestra.run.smithi202:> grep smithi202.front.sepia.ceph.com /etc/hosts 2023-06-22T17:58:48.398 INFO:teuthology.orchestra.run.smithi202.stdout:172.21.15.202 smithi202.front.sepia.ceph.com smithi202 2023-06-22T17:58:48.399 DEBUG:teuthology.orchestra.run.smithi202:> sudo hostname smithi202 2023-06-22T17:58:48.511 DEBUG:teuthology.orchestra.run.smithi202:> sudo sed -i -e 's/smithi202.front.sepia.ceph.com/smithi202/g' /etc/hosts 2023-06-22T17:58:48.536 DEBUG:teuthology.orchestra.run.smithi202:> sudo sed -i -e 's/172.21.15.202/172.21.15.202/g' /etc/hosts 2023-06-22T17:58:48.597 DEBUG:teuthology.orchestra.run.smithi202:> sudo sed -i -e 's/smithi202.front.sepia.ceph.com/smithi202/g' /etc/hostname 2023-06-22T17:58:48.660 DEBUG:teuthology.orchestra.run.smithi202:> sudo hostname smithi202 2023-06-22T17:58:48.723 DEBUG:teuthology.orchestra.run.smithi202:> cat /etc/os-release 2023-06-22T17:58:48.777 INFO:teuthology.orchestra.run.smithi202.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T17:58:48.777 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T17:58:48.777 INFO:teuthology.orchestra.run.smithi202.stdout:ID="rhel" 2023-06-22T17:58:48.777 INFO:teuthology.orchestra.run.smithi202.stdout:ID_LIKE="fedora" 2023-06-22T17:58:48.778 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION_ID="8.4" 2023-06-22T17:58:48.778 INFO:teuthology.orchestra.run.smithi202.stdout:PLATFORM_ID="platform:el8" 2023-06-22T17:58:48.778 INFO:teuthology.orchestra.run.smithi202.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T17:58:48.778 INFO:teuthology.orchestra.run.smithi202.stdout:ANSI_COLOR="0;31" 2023-06-22T17:58:48.778 INFO:teuthology.orchestra.run.smithi202.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T17:58:48.778 INFO:teuthology.orchestra.run.smithi202.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T17:58:48.778 INFO:teuthology.orchestra.run.smithi202.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T17:58:48.778 INFO:teuthology.orchestra.run.smithi202.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T17:58:48.779 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:58:48.779 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T17:58:48.779 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T17:58:48.779 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T17:58:48.779 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T17:58:48.779 INFO:teuthology.provision.fog.smithi202:Deploy complete! 2023-06-22T17:58:48.780 DEBUG:teuthology.parallel:result is None 2023-06-22T17:58:48.933 INFO:teuthology.lock.ops:Checking smithi047.front.sepia.ceph.com 2023-06-22T17:58:48.934 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-22T17:58:48.957 INFO:teuthology.lock.ops:Checking smithi202.front.sepia.ceph.com 2023-06-22T17:58:48.957 INFO:teuthology.lock.ops:New key found. Updating... 2023-06-22T17:58:48.983 INFO:teuthology.lock.ops:Updating [smithi047.front.sepia.ceph.com]: set os type and version on server 2023-06-22T17:58:48.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2023-06-22T17:58:48.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:58:49.049 DEBUG:teuthology.orchestra.run.smithi047:> true 2023-06-22T17:58:49.167 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2023-06-22T17:58:49.223 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2023-06-22T17:58:49.224 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2023-06-22T17:58:49.280 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T17:58:49.280 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T17:58:49.280 INFO:teuthology.orchestra.run.smithi047.stdout:ID="rhel" 2023-06-22T17:58:49.280 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="fedora" 2023-06-22T17:58:49.280 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="8.4" 2023-06-22T17:58:49.280 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el8" 2023-06-22T17:58:49.281 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T17:58:49.281 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2023-06-22T17:58:49.281 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T17:58:49.281 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T17:58:49.281 INFO:teuthology.orchestra.run.smithi047.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T17:58:49.281 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T17:58:49.281 INFO:teuthology.orchestra.run.smithi047.stdout: 2023-06-22T17:58:49.282 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T17:58:49.282 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T17:58:49.282 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T17:58:49.282 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T17:58:49.282 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2023-06-22T17:58:49.307 INFO:teuthology.lock.ops:Updating [smithi202.front.sepia.ceph.com]: set os type and version on server 2023-06-22T17:58:49.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi202.front.sepia.ceph.com' 2023-06-22T17:58:49.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi202.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-22T17:58:49.370 DEBUG:teuthology.orchestra.run.smithi202:> true 2023-06-22T17:58:49.434 DEBUG:teuthology.orchestra.run.smithi202:> uname -m 2023-06-22T17:58:49.488 INFO:teuthology.orchestra.run.smithi202.stdout:x86_64 2023-06-22T17:58:49.489 DEBUG:teuthology.orchestra.run.smithi202:> cat /etc/os-release 2023-06-22T17:58:49.542 INFO:teuthology.orchestra.run.smithi202.stdout:NAME="Red Hat Enterprise Linux" 2023-06-22T17:58:49.542 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION="8.4 (Ootpa)" 2023-06-22T17:58:49.542 INFO:teuthology.orchestra.run.smithi202.stdout:ID="rhel" 2023-06-22T17:58:49.542 INFO:teuthology.orchestra.run.smithi202.stdout:ID_LIKE="fedora" 2023-06-22T17:58:49.542 INFO:teuthology.orchestra.run.smithi202.stdout:VERSION_ID="8.4" 2023-06-22T17:58:49.543 INFO:teuthology.orchestra.run.smithi202.stdout:PLATFORM_ID="platform:el8" 2023-06-22T17:58:49.543 INFO:teuthology.orchestra.run.smithi202.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-06-22T17:58:49.543 INFO:teuthology.orchestra.run.smithi202.stdout:ANSI_COLOR="0;31" 2023-06-22T17:58:49.543 INFO:teuthology.orchestra.run.smithi202.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-06-22T17:58:49.543 INFO:teuthology.orchestra.run.smithi202.stdout:HOME_URL="https://www.redhat.com/" 2023-06-22T17:58:49.543 INFO:teuthology.orchestra.run.smithi202.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-22T17:58:49.543 INFO:teuthology.orchestra.run.smithi202.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-22T17:58:49.544 INFO:teuthology.orchestra.run.smithi202.stdout: 2023-06-22T17:58:49.544 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-22T17:58:49.544 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-06-22T17:58:49.544 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-22T17:58:49.544 INFO:teuthology.orchestra.run.smithi202.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-06-22T17:58:49.545 INFO:teuthology.lock.ops:Updating smithi202.front.sepia.ceph.com on lock server 2023-06-22T17:58:49.573 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T17:58:49.574 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T17:58:49.664 INFO:teuthology.dispatcher.supervisor:Running job 7311861 2023-06-22T17:58:49.665 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311861 --name yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi --description fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} kclient-overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts} tasks/kernel_cfuse_workunits_dbench_iozone} -- /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311861/orig.config.yaml 2023-06-22T17:58:49.669 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311861 2023-06-22T17:58:49.669 INFO:teuthology.dispatcher.supervisor:Job PID: 5533 2023-06-22T17:58:49.670 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-06-22T18:00:49.753 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T18:00:49.755 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T18:02:49.930 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T18:02:49.931 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T18:04:50.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T18:04:50.098 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T18:06:50.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T18:06:50.270 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T18:08:50.437 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T18:08:50.438 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T18:10:50.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T18:10:50.618 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T18:12:50.799 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-22T18:12:50.800 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-22T18:12:50.877 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-06-22T18:12:51.014 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-06-22T18:12:51.015 INFO:teuthology.nuke:Checking targets against current locks 2023-06-22T18:12:51.073 DEBUG:teuthology.nuke:shortname: smithi047 2023-06-22T18:12:51.074 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T18:12:51.104 DEBUG:teuthology.nuke:shortname: smithi202 2023-06-22T18:12:51.104 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-22T18:12:51.108 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311861', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 17:53:54.340599', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfitWEGuy8CvOJPmM/5bPelUSt5lE+oR8fa3Tp7Yq7Wu17JT/6yCi1kHd6k2Aw9reSl9Xj1j3WdGg/z5crRFEQuC8BfGMnpUUgaer9cMY367uF2Smn+cY83JZTQXT/OGz5Z0Qe2P3UbmtvG1SLiD6xOB9BRP7oq5350hxP03E4g7fz4wWovIOieYaV5ASEPr1HFm/siC+SbnLh/nlp0GWklvnhCXZpKla0ONzKLdzWhKcABoBfar9msT7tRo7BBFs8FeuSX9mrpk95sD5xLN1IKLI8eRCTXsaXT3wKxPeymbXzyjbECUkJQz7/+fOpLp5pSIVoymnJ+DsF6ZvH/Hjk9HJK/SO+jfVf09vNg/0M+HWHAlbIFEjm4d4j5TOhZI9BXI8R1sFk8UlHmrN56AOA142tjH8+KcYorYitIMe604S4EkrFZvklvEPQHsfET4kGhBUB17yohBaDyTdduODpkYqggm6w47fNC9CbcwS4T11Q/5XCR+Qlop+cgDioH40='} 2023-06-22T18:12:51.130 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi202.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-22_15:25:25-fs-wip-yuri4-testing-2023-06-21-0846-quincy-distro-default-smithi/7311861', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-22 17:53:54.342069', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCynzxqKL7f4FSFj7DJAfPsbEty8jErAkIkRt28/jOQWGkousBsN1JGnbMrHhVW6oTkRczEbKlIcfjHXxeKnEOiETs5h0KhL41QjT19PIpdm4i1SMbUbUaj0zPZ1FqAu+cYHrMQpWVSzOZCMtkdaPkJm3U1tI2RgyPzc1IzIZwzRw5b7A0VOwMNzVAx8HdQ99MQA2CDG2bWuHmHsAskWMUFICXc23dna5EcTVBpja1OCEbkEinCxau01ep7D/zKR0spIEl/vaS3t9OLwAn9f467YAv/sjnXQMW+w3W7AOYLP/WKJoTSgDiYLFhOb5K/qN6qWF74ACiXWOv5MavE8OeUQNZzypLmG98A9pxdH7Cp85CdkNDomAkfUcsxQqncvmPUr92wckSYXUA6Jcb9/ZLttxEuVWJykHkeujjxsemKhu20hdhDqaSD/FmCwb5Ok2//0nvKc58RX8lznSLx+g/36clSCAn+zbJoRfnpCW+Q+XHVlmwPichGnTSEWWTmV68='} 2023-06-22T18:12:51.162 INFO:teuthology.orchestra.console:Power off smithi047 2023-06-22T18:12:51.163 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T18:12:51.186 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi047.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:12:51.205 INFO:teuthology.orchestra.console:Power off smithi202 2023-06-22T18:12:51.206 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-22T18:12:51.228 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi202.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-22T18:12:51.318 INFO:teuthology.orchestra.console:Power off for smithi047 completed 2023-06-22T18:12:51.349 INFO:teuthology.orchestra.console:Power off for smithi202 completed 2023-06-22T18:12:51.483 INFO:teuthology.lock.ops:unlocked: smithi047.front.sepia.ceph.com 2023-06-22T18:12:51.500 INFO:teuthology.lock.ops:unlocked: smithi202.front.sepia.ceph.com