2023-07-21T01:05:24.021 INFO:root:teuthology version: 0.0.1.dev153+g79491cb
2023-07-21T01:05:24.022 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:05:24.022 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:05:24.183 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f05dae3b580>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f05daea5e50>, signals=[15])
2023-07-21T01:05:24.185 INFO:teuthology.lock.ops:Start node 'smithi169.front.sepia.ceph.com' reimaging
2023-07-21T01:05:24.185 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: reset os type and version on server
2023-07-21T01:05:24.185 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2023-07-21T01:05:24.206 INFO:teuthology.lock.ops:Start node 'smithi192.front.sepia.ceph.com' reimaging
2023-07-21T01:05:24.207 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: reset os type and version on server
2023-07-21T01:05:24.207 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server
2023-07-21T01:05:24.208 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /host {"name": "smithi169"}
2023-07-21T01:05:24.232 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /host {"name": "smithi192"}
2023-07-21T01:05:24.285 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-07-21T01:05:24.303 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /image {"name": "smithi_centos_9.stream"}
2023-07-21T01:05:24.328 DEBUG:teuthology.provision.fog.smithi169:Requesting image smithi_centos_9.stream (ID 10)
2023-07-21T01:05:24.329 DEBUG:teuthology.provision.fog.smithi169:FOG request: PUT /host/274 {"imageID": 10}
2023-07-21T01:05:24.346 DEBUG:teuthology.provision.fog.smithi192:Requesting image smithi_centos_9.stream (ID 10)
2023-07-21T01:05:24.347 DEBUG:teuthology.provision.fog.smithi192:FOG request: PUT /host/297 {"imageID": 10}
2023-07-21T01:05:24.372 INFO:teuthology.provision.fog.smithi169:Scheduling deploy of centos 9.stream
2023-07-21T01:05:24.373 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:05:24.387 INFO:teuthology.provision.fog.smithi192:Scheduling deploy of centos 9.stream
2023-07-21T01:05:24.387 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:05:24.426 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /tasktype {"name": "deploy"}
2023-07-21T01:05:24.436 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /tasktype {"name": "deploy"}
2023-07-21T01:05:24.460 DEBUG:teuthology.provision.fog.smithi169:FOG request: POST /host/274/task {"taskTypeID": 1}
2023-07-21T01:05:24.477 DEBUG:teuthology.provision.fog.smithi192:FOG request: POST /host/297/task {"taskTypeID": 1}
2023-07-21T01:05:24.533 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:05:24.542 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:05:24.628 INFO:teuthology.orchestra.console:Power off smithi169
2023-07-21T01:05:24.629 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-21T01:05:24.642 INFO:teuthology.orchestra.console:Power off smithi192
2023-07-21T01:05:24.642 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-21T01:05:24.650 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:24.661 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:28.672 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:28.681 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:32.784 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:32.794 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:36.896 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:36.906 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:37.108 INFO:teuthology.orchestra.console:Power off for smithi169 completed
2023-07-21T01:05:37.115 INFO:teuthology.orchestra.console:Power off for smithi192 completed
2023-07-21T01:05:37.210 INFO:teuthology.orchestra.console:Power on smithi169
2023-07-21T01:05:37.210 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-21T01:05:37.218 INFO:teuthology.orchestra.console:Power on smithi192
2023-07-21T01:05:37.219 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-21T01:05:37.230 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:37.239 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:41.252 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:41.260 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:41.471 INFO:teuthology.orchestra.console:Power on for smithi192 completed
2023-07-21T01:05:41.572 INFO:teuthology.provision.fog.smithi192:Waiting for deploy to finish
2023-07-21T01:05:41.572 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:05:45.366 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T01:05:45.577 INFO:teuthology.orchestra.console:Power on for smithi169 completed
2023-07-21T01:05:45.678 INFO:teuthology.provision.fog.smithi169:Waiting for deploy to finish
2023-07-21T01:05:45.678 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:05:56.647 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:06:00.744 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:06:11.728 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:06:15.800 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:06:26.797 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:06:30.874 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:06:41.866 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:06:45.940 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:06:56.936 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:07:01.012 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:07:12.005 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:07:16.092 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:07:27.087 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:07:31.160 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:07:42.168 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:07:46.235 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:07:57.249 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:08:01.310 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:08:12.326 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:08:16.388 DEBUG:teuthology.provision.fog.smithi169:FOG request: GET /task/active None
2023-07-21T01:08:16.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:08:16.546 ERROR:teuthology.orchestra.connection:Error authenticating with smithi169.front.sepia.ceph.com: Authentication failed.
2023-07-21T01:08:16.547 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-21T01:08:16.548 DEBUG:teuthology.orchestra.run.smithi169:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-21T01:08:16.549 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-07-21T01:08:27.405 DEBUG:teuthology.provision.fog.smithi192:FOG request: GET /task/active None
2023-07-21T01:08:27.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:08:30.544 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192
2023-07-21T01:08:36.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:08:39.600 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192
2023-07-21T01:08:45.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:08:48.656 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192
2023-07-21T01:08:54.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:08:57.712 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192
2023-07-21T01:09:03.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:09:06.768 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192
2023-07-21T01:09:12.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:09:15.824 WARNING:teuthology.provision.fog.smithi192:[Errno None] Unable to connect to port 22 on 172.21.15.192
2023-07-21T01:09:21.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:10:21.828 WARNING:teuthology.provision.fog.smithi192:timed out
2023-07-21T01:10:27.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:10:28.105 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-21T01:10:28.106 DEBUG:teuthology.orchestra.run.smithi192:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-21T01:10:28.546 INFO:teuthology.provision.fog.smithi192:Node is ready
2023-07-21T01:10:28.547 DEBUG:teuthology.orchestra.run.smithi192:> hostname
2023-07-21T01:10:28.563 INFO:teuthology.orchestra.run.smithi192.stdout:smithi192.front.sepia.ceph.com
2023-07-21T01:10:28.564 DEBUG:teuthology.orchestra.run.smithi192:> grep smithi192.front.sepia.ceph.com /etc/hosts
2023-07-21T01:10:28.618 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.15.192 smithi192.front.sepia.ceph.com smithi192
2023-07-21T01:10:28.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname smithi192
2023-07-21T01:10:28.731 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/smithi192.front.sepia.ceph.com/smithi192/g' /etc/hosts
2023-07-21T01:10:28.759 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/172.21.15.192/172.21.15.192/g' /etc/hosts
2023-07-21T01:10:28.828 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i -e 's/smithi192.front.sepia.ceph.com/smithi192/g' /etc/hostname
2023-07-21T01:10:28.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname smithi192
2023-07-21T01:10:28.962 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release
2023-07-21T01:10:29.016 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream"
2023-07-21T01:10:29.017 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9"
2023-07-21T01:10:29.017 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos"
2023-07-21T01:10:29.017 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora"
2023-07-21T01:10:29.017 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9"
2023-07-21T01:10:29.017 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9"
2023-07-21T01:10:29.017 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9"
2023-07-21T01:10:29.017 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31"
2023-07-21T01:10:29.018 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon"
2023-07-21T01:10:29.018 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-07-21T01:10:29.018 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/"
2023-07-21T01:10:29.018 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-21T01:10:29.018 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-07-21T01:10:29.018 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-21T01:10:29.019 INFO:teuthology.provision.fog.smithi192:Deploy complete!
2023-07-21T01:10:29.020 DEBUG:teuthology.parallel:result is None
2023-07-21T01:10:34.356 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-07-21T01:10:34.357 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi169
2023-07-21T01:10:43.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2023-07-21T01:10:43.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:10:43.567 DEBUG:teuthology.orchestra.run.smithi169:> true
2023-07-21T01:10:43.916 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com'
2023-07-21T01:10:43.917 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-21T01:10:43.917 DEBUG:teuthology.orchestra.run.smithi169:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-21T01:10:43.931 INFO:teuthology.provision.fog.smithi169:Node is ready
2023-07-21T01:10:43.931 DEBUG:teuthology.orchestra.run.smithi169:> hostname
2023-07-21T01:10:43.986 INFO:teuthology.orchestra.run.smithi169.stdout:smithi169.front.sepia.ceph.com
2023-07-21T01:10:43.986 DEBUG:teuthology.orchestra.run.smithi169:> grep smithi169.front.sepia.ceph.com /etc/hosts
2023-07-21T01:10:44.040 INFO:teuthology.orchestra.run.smithi169.stdout:172.21.15.169 smithi169.front.sepia.ceph.com smithi169
2023-07-21T01:10:44.040 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169
2023-07-21T01:10:44.167 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hosts
2023-07-21T01:10:44.196 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/172.21.15.169/172.21.15.169/g' /etc/hosts
2023-07-21T01:10:44.262 DEBUG:teuthology.orchestra.run.smithi169:> sudo sed -i -e 's/smithi169.front.sepia.ceph.com/smithi169/g' /etc/hostname
2023-07-21T01:10:44.329 DEBUG:teuthology.orchestra.run.smithi169:> sudo hostname smithi169
2023-07-21T01:10:44.396 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release
2023-07-21T01:10:44.451 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream"
2023-07-21T01:10:44.451 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9"
2023-07-21T01:10:44.451 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos"
2023-07-21T01:10:44.451 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora"
2023-07-21T01:10:44.451 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9"
2023-07-21T01:10:44.452 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9"
2023-07-21T01:10:44.452 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9"
2023-07-21T01:10:44.452 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2023-07-21T01:10:44.452 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon"
2023-07-21T01:10:44.452 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-07-21T01:10:44.452 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/"
2023-07-21T01:10:44.452 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-21T01:10:44.453 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-07-21T01:10:44.453 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-21T01:10:44.453 INFO:teuthology.provision.fog.smithi169:Deploy complete!
2023-07-21T01:10:44.454 DEBUG:teuthology.parallel:result is None
2023-07-21T01:10:44.610 INFO:teuthology.lock.ops:Checking smithi169.front.sepia.ceph.com
2023-07-21T01:10:44.611 INFO:teuthology.lock.ops:New key found. Updating...
2023-07-21T01:10:44.639 INFO:teuthology.lock.ops:Checking smithi192.front.sepia.ceph.com
2023-07-21T01:10:44.639 INFO:teuthology.lock.ops:New key found. Updating...
2023-07-21T01:10:44.665 INFO:teuthology.lock.ops:Updating [smithi169.front.sepia.ceph.com]: set os type and version on server
2023-07-21T01:10:44.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi169.front.sepia.ceph.com'
2023-07-21T01:10:44.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi169.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:10:44.748 DEBUG:teuthology.orchestra.run.smithi169:> true
2023-07-21T01:10:44.818 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi169.front.sepia.ceph.com'
2023-07-21T01:10:44.818 DEBUG:teuthology.orchestra.run.smithi169:> uname -m
2023-07-21T01:10:44.872 INFO:teuthology.orchestra.run.smithi169.stdout:x86_64
2023-07-21T01:10:44.873 DEBUG:teuthology.orchestra.run.smithi169:> cat /etc/os-release
2023-07-21T01:10:44.927 INFO:teuthology.orchestra.run.smithi169.stdout:NAME="CentOS Stream"
2023-07-21T01:10:44.927 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION="9"
2023-07-21T01:10:44.927 INFO:teuthology.orchestra.run.smithi169.stdout:ID="centos"
2023-07-21T01:10:44.927 INFO:teuthology.orchestra.run.smithi169.stdout:ID_LIKE="rhel fedora"
2023-07-21T01:10:44.928 INFO:teuthology.orchestra.run.smithi169.stdout:VERSION_ID="9"
2023-07-21T01:10:44.928 INFO:teuthology.orchestra.run.smithi169.stdout:PLATFORM_ID="platform:el9"
2023-07-21T01:10:44.928 INFO:teuthology.orchestra.run.smithi169.stdout:PRETTY_NAME="CentOS Stream 9"
2023-07-21T01:10:44.928 INFO:teuthology.orchestra.run.smithi169.stdout:ANSI_COLOR="0;31"
2023-07-21T01:10:44.928 INFO:teuthology.orchestra.run.smithi169.stdout:LOGO="fedora-logo-icon"
2023-07-21T01:10:44.928 INFO:teuthology.orchestra.run.smithi169.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-07-21T01:10:44.928 INFO:teuthology.orchestra.run.smithi169.stdout:HOME_URL="https://centos.org/"
2023-07-21T01:10:44.929 INFO:teuthology.orchestra.run.smithi169.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-21T01:10:44.929 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-07-21T01:10:44.929 INFO:teuthology.orchestra.run.smithi169.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-21T01:10:44.929 INFO:teuthology.lock.ops:Updating smithi169.front.sepia.ceph.com on lock server
2023-07-21T01:10:44.956 INFO:teuthology.lock.ops:Updating [smithi192.front.sepia.ceph.com]: set os type and version on server
2023-07-21T01:10:44.956 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com'
2023-07-21T01:10:44.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-21T01:10:45.041 DEBUG:teuthology.orchestra.run.smithi192:> true
2023-07-21T01:10:45.113 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com'
2023-07-21T01:10:45.113 DEBUG:teuthology.orchestra.run.smithi192:> uname -m
2023-07-21T01:10:45.169 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64
2023-07-21T01:10:45.169 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release
2023-07-21T01:10:45.225 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream"
2023-07-21T01:10:45.225 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9"
2023-07-21T01:10:45.225 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos"
2023-07-21T01:10:45.225 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora"
2023-07-21T01:10:45.226 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9"
2023-07-21T01:10:45.226 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9"
2023-07-21T01:10:45.226 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9"
2023-07-21T01:10:45.227 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31"
2023-07-21T01:10:45.227 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon"
2023-07-21T01:10:45.227 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2023-07-21T01:10:45.227 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/"
2023-07-21T01:10:45.227 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-21T01:10:45.227 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2023-07-21T01:10:45.228 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-07-21T01:10:45.228 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server
2023-07-21T01:10:45.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:10:45.256 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:10:45.372 INFO:teuthology.dispatcher.supervisor:Running job 7345573
2023-07-21T01:10:45.372 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/virtualenv/bin/teuthology -v --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2023-07-20_23:54:45-rgw-wip-cbodley-testing-distro-default-smithi/7345573 --name cbodley-2023-07-20_23:54:45-rgw-wip-cbodley-testing-distro-default-smithi --description rgw/verify/{0-install clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} -- /home/teuthworker/archive/cbodley-2023-07-20_23:54:45-rgw-wip-cbodley-testing-distro-default-smithi/7345573/orig.config.yaml
2023-07-21T01:10:45.376 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/cbodley-2023-07-20_23:54:45-rgw-wip-cbodley-testing-distro-default-smithi/7345573
2023-07-21T01:10:45.377 INFO:teuthology.dispatcher.supervisor:Job PID: 29624
2023-07-21T01:10:45.377 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-07-21T01:12:45.476 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:12:45.478 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:14:45.664 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:14:45.665 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:16:45.844 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:16:45.845 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:18:46.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:18:46.025 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:20:46.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:20:46.196 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:22:46.361 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:22:46.362 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:24:46.541 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:24:46.542 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:26:46.721 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:26:46.722 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:28:46.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:28:46.894 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:30:47.069 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:30:47.070 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:32:47.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:32:47.225 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:34:47.393 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:34:47.394 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:36:47.565 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:36:47.566 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:38:47.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:38:47.733 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:40:47.893 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:40:47.894 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:42:48.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:42:48.063 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:44:48.232 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:44:48.232 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:46:48.396 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:46:48.397 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:48:48.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:48:48.568 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:50:48.753 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:50:48.753 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:52:48.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:52:48.929 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:54:49.102 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:54:49.103 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:56:49.273 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:56:49.274 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T01:58:49.441 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T01:58:49.442 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:00:49.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T02:00:49.610 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:02:49.786 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T02:02:49.787 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:04:49.961 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T02:04:49.961 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:06:50.125 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T02:06:50.126 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:08:50.293 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T02:08:50.294 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:10:50.464 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T02:10:50.465 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:12:53.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T02:12:53.977 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:14:59.153 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-21T02:14:59.154 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-21T02:15:09.092 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-07-21T02:15:19.290 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-07-21T02:15:19.290 INFO:teuthology.nuke:Checking targets against current locks
2023-07-21T02:15:39.031 DEBUG:teuthology.nuke:shortname: smithi169
2023-07-21T02:15:39.032 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-21T02:15:49.072 DEBUG:teuthology.nuke:shortname: smithi192
2023-07-21T02:15:49.072 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-21T02:15:54.078 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi169.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-07-20_23:54:45-rgw-wip-cbodley-testing-distro-default-smithi/7345573', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 01:05:23.198992', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNpJiwI0tSZMbjXqpfYS1Xp/AwVVl+lBRQ136ly8dtswS89b3prhOWvf/pVwSP1d6yM+9/L8StLAMBf/VNmDQ0w='}
2023-07-21T02:15:59.113 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2023-07-20_23:54:45-rgw-wip-cbodley-testing-distro-default-smithi/7345573', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 01:05:23.201231', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDqxIyEAUmVrKLjZYiyeHwqLex85bl93WYikSvDgcw4qIPRz0aTVlxQ/jFJQom0+YmKJyt2BG6KsT3rqUEDb9UM='}
2023-07-21T02:16:14.167 INFO:teuthology.orchestra.console:Power off smithi169
2023-07-21T02:16:14.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-21T02:16:14.212 INFO:teuthology.orchestra.console:Power off smithi192
2023-07-21T02:16:14.212 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-21T02:16:19.194 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T02:16:19.239 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T02:16:24.208 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi169.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T02:16:24.251 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-21T02:16:24.463 INFO:teuthology.orchestra.console:Power off for smithi192 completed
2023-07-21T02:16:34.429 INFO:teuthology.orchestra.console:Power off for smithi169 completed
2023-07-21T02:16:34.465 INFO:teuthology.lock.ops:unlocked: smithi192.front.sepia.ceph.com
2023-07-21T02:16:39.512 INFO:teuthology.lock.ops:unlocked: smithi169.front.sepia.ceph.com