2023-07-14T18:49:08.210 INFO:root:teuthology version: 0.0.1.dev145+gaa6a80e
2023-07-14T18:49:08.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T18:49:08.212 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T18:49:08.408 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f2eefaaf5e0>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f2ef0cd58b0>, signals=[15])
2023-07-14T18:49:08.410 INFO:teuthology.lock.ops:Start node 'smithi053.front.sepia.ceph.com' reimaging
2023-07-14T18:49:08.410 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: reset os type and version on server
2023-07-14T18:49:08.410 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server
2023-07-14T18:49:08.430 INFO:teuthology.lock.ops:Start node 'smithi070.front.sepia.ceph.com' reimaging
2023-07-14T18:49:08.431 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: reset os type and version on server
2023-07-14T18:49:08.431 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2023-07-14T18:49:08.432 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /host {"name": "smithi053"}
2023-07-14T18:49:09.313 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging
2023-07-14T18:49:09.314 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server
2023-07-14T18:49:09.314 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2023-07-14T18:49:09.315 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /host {"name": "smithi070"}
2023-07-14T18:49:09.330 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /host {"name": "smithi104"}
2023-07-14T18:49:09.365 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-07-14T18:49:09.441 DEBUG:teuthology.provision.fog.smithi053:Requesting image smithi_rhel_8.6 (ID 5)
2023-07-14T18:49:09.441 DEBUG:teuthology.provision.fog.smithi053:FOG request: PUT /host/158 {"imageID": 5}
2023-07-14T18:49:09.460 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-07-14T18:49:09.465 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-07-14T18:49:09.478 INFO:teuthology.provision.fog.smithi053:Scheduling deploy of rhel 8.6
2023-07-14T18:49:09.479 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None
2023-07-14T18:49:09.502 DEBUG:teuthology.provision.fog.smithi104:Requesting image smithi_rhel_8.6 (ID 5)
2023-07-14T18:49:09.502 DEBUG:teuthology.provision.fog.smithi104:FOG request: PUT /host/209 {"imageID": 5}
2023-07-14T18:49:09.507 DEBUG:teuthology.provision.fog.smithi070:Requesting image smithi_rhel_8.6 (ID 5)
2023-07-14T18:49:09.507 DEBUG:teuthology.provision.fog.smithi070:FOG request: PUT /host/175 {"imageID": 5}
2023-07-14T18:49:09.539 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /tasktype {"name": "deploy"}
2023-07-14T18:49:09.542 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of rhel 8.6
2023-07-14T18:49:09.542 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:49:09.557 INFO:teuthology.provision.fog.smithi070:Scheduling deploy of rhel 8.6
2023-07-14T18:49:09.558 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:49:09.583 DEBUG:teuthology.provision.fog.smithi053:FOG request: POST /host/158/task {"taskTypeID": 1}
2023-07-14T18:49:09.613 DEBUG:teuthology.provision.fog.smithi104:Canceling deploy task with ID 384884
2023-07-14T18:49:09.614 DEBUG:teuthology.provision.fog.smithi104:FOG request: DELETE /task/cancel {"id": 384884}
2023-07-14T18:49:09.616 DEBUG:teuthology.provision.fog.smithi070:Canceling deploy task with ID 384883
2023-07-14T18:49:09.617 DEBUG:teuthology.provision.fog.smithi070:FOG request: DELETE /task/cancel {"id": 384883}
2023-07-14T18:49:09.642 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None
2023-07-14T18:49:09.665 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /tasktype {"name": "deploy"}
2023-07-14T18:49:09.668 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /tasktype {"name": "deploy"}
2023-07-14T18:49:09.707 DEBUG:teuthology.provision.fog.smithi104:FOG request: POST /host/209/task {"taskTypeID": 1}
2023-07-14T18:49:09.710 DEBUG:teuthology.provision.fog.smithi070:FOG request: POST /host/175/task {"taskTypeID": 1}
2023-07-14T18:49:09.736 INFO:teuthology.orchestra.console:Power off smithi053
2023-07-14T18:49:09.737 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-14T18:49:09.756 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:49:09.759 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:09.767 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:49:09.850 INFO:teuthology.orchestra.console:Power off smithi104
2023-07-14T18:49:09.851 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-14T18:49:09.863 INFO:teuthology.orchestra.console:Power off smithi070
2023-07-14T18:49:09.864 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-14T18:49:09.872 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:09.881 INFO:teuthology.orchestra.console:Power off for smithi053 completed
2023-07-14T18:49:09.883 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:09.982 INFO:teuthology.orchestra.console:Power on smithi053
2023-07-14T18:49:09.983 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-14T18:49:09.995 INFO:teuthology.orchestra.console:Power off for smithi104 completed
2023-07-14T18:49:10.004 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2023-07-14T18:49:10.006 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:10.097 INFO:teuthology.orchestra.console:Power on smithi104
2023-07-14T18:49:10.098 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-14T18:49:10.105 INFO:teuthology.orchestra.console:Power on smithi070
2023-07-14T18:49:10.106 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-07-14T18:49:10.118 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:10.126 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:14.028 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:14.139 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:14.146 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:19.139 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:19.249 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:19.257 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:24.252 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:24.360 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:24.368 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:29.371 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:29.469 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:29.479 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:34.492 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:34.580 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:34.589 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:34.799 INFO:teuthology.orchestra.console:Power on for smithi070 completed
2023-07-14T18:49:34.900 INFO:teuthology.provision.fog.smithi070:Waiting for deploy to finish
2023-07-14T18:49:34.901 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:49:39.603 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:39.691 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T18:49:39.814 INFO:teuthology.orchestra.console:Power on for smithi053 completed
2023-07-14T18:49:39.901 INFO:teuthology.orchestra.console:Power on for smithi104 completed
2023-07-14T18:49:39.916 INFO:teuthology.provision.fog.smithi053:Waiting for deploy to finish
2023-07-14T18:49:39.916 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None
2023-07-14T18:49:40.003 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish
2023-07-14T18:49:40.003 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:49:49.969 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:49:58.089 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None
2023-07-14T18:49:58.090 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:50:13.170 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:50:18.177 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None
2023-07-14T18:50:18.178 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:50:35.616 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:50:35.618 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None
2023-07-14T18:50:45.631 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:51:05.652 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:51:10.657 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:51:10.659 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None
2023-07-14T18:51:25.788 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:51:25.789 DEBUG:teuthology.provision.fog.smithi053:FOG request: GET /task/active None
2023-07-14T18:51:25.790 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:51:45.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:51:48.976 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-07-14T18:51:54.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:00.900 DEBUG:teuthology.provision.fog.smithi104:FOG request: GET /task/active None
2023-07-14T18:52:00.902 DEBUG:teuthology.provision.fog.smithi070:FOG request: GET /task/active None
2023-07-14T18:52:09.748 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-07-14T18:52:09.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:09.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:12.944 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2023-07-14T18:52:12.980 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-14T18:52:15.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:18.800 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-07-14T18:52:18.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:23.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:28.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:33.960 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2023-07-14T18:52:33.960 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-14T18:52:39.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:44.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:48.048 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104
2023-07-14T18:52:48.049 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-14T18:52:52.464 WARNING:teuthology.provision.fog.smithi053:[Errno None] Unable to connect to port 22 on 172.21.15.53
2023-07-14T18:52:54.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:54.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:52:58.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:53:03.823 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-14T18:53:03.824 DEBUG:teuthology.orchestra.run.smithi053:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-14T18:53:04.072 INFO:teuthology.provision.fog.smithi053:Node is ready
2023-07-14T18:53:04.072 DEBUG:teuthology.orchestra.run.smithi053:> hostname
2023-07-14T18:53:04.131 INFO:teuthology.orchestra.run.smithi053.stdout:smithi053.front.sepia.ceph.com
2023-07-14T18:53:04.132 DEBUG:teuthology.orchestra.run.smithi053:> grep smithi053.front.sepia.ceph.com /etc/hosts
2023-07-14T18:53:04.191 INFO:teuthology.orchestra.run.smithi053.stdout:172.21.15.53 smithi053.front.sepia.ceph.com smithi053
2023-07-14T18:53:04.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053
2023-07-14T18:53:04.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hosts
2023-07-14T18:53:04.353 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/172.21.15.53/172.21.15.53/g' /etc/hosts
2023-07-14T18:53:04.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i -e 's/smithi053.front.sepia.ceph.com/smithi053/g' /etc/hostname
2023-07-14T18:53:04.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo hostname smithi053
2023-07-14T18:53:04.565 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release
2023-07-14T18:53:04.625 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux"
2023-07-14T18:53:04.625 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.6 (Ootpa)"
2023-07-14T18:53:04.626 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel"
2023-07-14T18:53:04.626 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora"
2023-07-14T18:53:04.626 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.6"
2023-07-14T18:53:04.626 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8"
2023-07-14T18:53:04.626 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-14T18:53:04.626 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31"
2023-07-14T18:53:04.627 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-14T18:53:04.627 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/"
2023-07-14T18:53:04.627 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-14T18:53:04.627 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-14T18:53:04.627 INFO:teuthology.orchestra.run.smithi053.stdout:
2023-07-14T18:53:04.627 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-14T18:53:04.627 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-14T18:53:04.628 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-14T18:53:04.628 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-14T18:53:04.628 INFO:teuthology.provision.fog.smithi053:Deploy complete!
2023-07-14T18:53:04.630 DEBUG:teuthology.parallel:result is None
2023-07-14T18:53:12.688 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-14T18:53:18.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:53:21.748 WARNING:teuthology.provision.fog.smithi070:[Errno None] Unable to connect to port 22 on 172.21.15.70
2023-07-14T18:53:27.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:53:33.083 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-14T18:53:33.083 DEBUG:teuthology.orchestra.run.smithi070:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-14T18:53:33.359 INFO:teuthology.provision.fog.smithi070:Node is ready
2023-07-14T18:53:33.360 DEBUG:teuthology.orchestra.run.smithi070:> hostname
2023-07-14T18:53:33.375 INFO:teuthology.orchestra.run.smithi070.stdout:smithi070.front.sepia.ceph.com
2023-07-14T18:53:33.376 DEBUG:teuthology.orchestra.run.smithi070:> grep smithi070.front.sepia.ceph.com /etc/hosts
2023-07-14T18:53:33.431 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.70 smithi070.front.sepia.ceph.com smithi070
2023-07-14T18:53:33.431 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname smithi070
2023-07-14T18:53:33.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/smithi070.front.sepia.ceph.com/smithi070/g' /etc/hosts
2023-07-14T18:53:33.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/172.21.15.70/172.21.15.70/g' /etc/hosts
2023-07-14T18:53:33.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i -e 's/smithi070.front.sepia.ceph.com/smithi070/g' /etc/hostname
2023-07-14T18:53:33.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname smithi070
2023-07-14T18:53:33.783 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release
2023-07-14T18:53:33.839 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Red Hat Enterprise Linux"
2023-07-14T18:53:33.839 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8.6 (Ootpa)"
2023-07-14T18:53:33.839 INFO:teuthology.orchestra.run.smithi070.stdout:ID="rhel"
2023-07-14T18:53:33.839 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="fedora"
2023-07-14T18:53:33.840 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8.6"
2023-07-14T18:53:33.840 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8"
2023-07-14T18:53:33.840 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-14T18:53:33.840 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31"
2023-07-14T18:53:33.840 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-14T18:53:33.840 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.redhat.com/"
2023-07-14T18:53:33.840 INFO:teuthology.orchestra.run.smithi070.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-14T18:53:33.841 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-14T18:53:33.841 INFO:teuthology.orchestra.run.smithi070.stdout:
2023-07-14T18:53:33.841 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-14T18:53:33.841 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-14T18:53:33.841 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-14T18:53:33.841 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-14T18:53:33.842 INFO:teuthology.provision.fog.smithi070:Deploy complete!
2023-07-14T18:53:33.842 DEBUG:teuthology.parallel:result is None
2023-07-14T18:53:54.280 WARNING:teuthology.provision.fog.smithi104:timed out
2023-07-14T18:54:00.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:54:00.523 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-07-14T18:54:00.524 DEBUG:teuthology.orchestra.run.smithi104:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-07-14T18:54:00.835 INFO:teuthology.provision.fog.smithi104:Node is ready
2023-07-14T18:54:00.835 DEBUG:teuthology.orchestra.run.smithi104:> hostname
2023-07-14T18:54:00.853 INFO:teuthology.orchestra.run.smithi104.stdout:smithi104.front.sepia.ceph.com
2023-07-14T18:54:00.853 DEBUG:teuthology.orchestra.run.smithi104:> grep smithi104.front.sepia.ceph.com /etc/hosts
2023-07-14T18:54:00.911 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.104 smithi104.front.sepia.ceph.com smithi104
2023-07-14T18:54:00.911 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104
2023-07-14T18:54:01.019 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hosts
2023-07-14T18:54:01.046 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/172.21.15.104/172.21.15.104/g' /etc/hosts
2023-07-14T18:54:01.113 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hostname
2023-07-14T18:54:01.179 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104
2023-07-14T18:54:01.245 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release
2023-07-14T18:54:01.301 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Red Hat Enterprise Linux"
2023-07-14T18:54:01.302 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="8.6 (Ootpa)"
2023-07-14T18:54:01.302 INFO:teuthology.orchestra.run.smithi104.stdout:ID="rhel"
2023-07-14T18:54:01.302 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="fedora"
2023-07-14T18:54:01.302 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="8.6"
2023-07-14T18:54:01.302 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el8"
2023-07-14T18:54:01.302 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-14T18:54:01.303 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31"
2023-07-14T18:54:01.303 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-14T18:54:01.303 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.redhat.com/"
2023-07-14T18:54:01.303 INFO:teuthology.orchestra.run.smithi104.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-14T18:54:01.303 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-14T18:54:01.303 INFO:teuthology.orchestra.run.smithi104.stdout:
2023-07-14T18:54:01.303 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-14T18:54:01.303 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-14T18:54:01.304 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-14T18:54:01.304 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-14T18:54:01.304 INFO:teuthology.provision.fog.smithi104:Deploy complete!
2023-07-14T18:54:01.305 DEBUG:teuthology.parallel:result is None
2023-07-14T18:54:16.517 INFO:teuthology.lock.ops:Checking smithi053.front.sepia.ceph.com
2023-07-14T18:54:16.518 INFO:teuthology.lock.ops:Checking smithi070.front.sepia.ceph.com
2023-07-14T18:54:16.518 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com
2023-07-14T18:54:16.520 INFO:teuthology.lock.ops:Updating [smithi053.front.sepia.ceph.com]: set os type and version on server
2023-07-14T18:54:16.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com'
2023-07-14T18:54:16.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:54:16.589 DEBUG:teuthology.orchestra.run.smithi053:> true
2023-07-14T18:54:16.722 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com'
2023-07-14T18:54:16.755 DEBUG:teuthology.orchestra.run.smithi053:> uname -m
2023-07-14T18:54:16.780 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64
2023-07-14T18:54:16.780 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release
2023-07-14T18:54:16.838 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="Red Hat Enterprise Linux"
2023-07-14T18:54:16.838 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="8.6 (Ootpa)"
2023-07-14T18:54:16.838 INFO:teuthology.orchestra.run.smithi053.stdout:ID="rhel"
2023-07-14T18:54:16.838 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="fedora"
2023-07-14T18:54:16.839 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="8.6"
2023-07-14T18:54:16.839 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el8"
2023-07-14T18:54:16.839 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-14T18:54:16.839 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31"
2023-07-14T18:54:16.839 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-14T18:54:16.839 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://www.redhat.com/"
2023-07-14T18:54:16.839 INFO:teuthology.orchestra.run.smithi053.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-14T18:54:16.840 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-14T18:54:16.840 INFO:teuthology.orchestra.run.smithi053.stdout:
2023-07-14T18:54:16.840 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-14T18:54:16.840 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-14T18:54:16.840 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-14T18:54:16.840 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-14T18:54:16.841 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server
2023-07-14T18:54:16.868 INFO:teuthology.lock.ops:Updating [smithi070.front.sepia.ceph.com]: set os type and version on server
2023-07-14T18:54:16.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com'
2023-07-14T18:54:16.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:54:16.935 DEBUG:teuthology.orchestra.run.smithi070:> true
2023-07-14T18:54:17.070 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com'
2023-07-14T18:54:17.070 DEBUG:teuthology.orchestra.run.smithi070:> uname -m
2023-07-14T18:54:17.127 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64
2023-07-14T18:54:17.128 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release
2023-07-14T18:54:17.183 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="Red Hat Enterprise Linux"
2023-07-14T18:54:17.184 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="8.6 (Ootpa)"
2023-07-14T18:54:17.184 INFO:teuthology.orchestra.run.smithi070.stdout:ID="rhel"
2023-07-14T18:54:17.184 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="fedora"
2023-07-14T18:54:17.184 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="8.6"
2023-07-14T18:54:17.185 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el8"
2023-07-14T18:54:17.185 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-14T18:54:17.185 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31"
2023-07-14T18:54:17.185 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-14T18:54:17.185 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://www.redhat.com/"
2023-07-14T18:54:17.185 INFO:teuthology.orchestra.run.smithi070.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-14T18:54:17.185 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-14T18:54:17.186 INFO:teuthology.orchestra.run.smithi070.stdout:
2023-07-14T18:54:17.186 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-14T18:54:17.186 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-14T18:54:17.186 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-14T18:54:17.186 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-14T18:54:17.187 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server
2023-07-14T18:54:17.210 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server
2023-07-14T18:54:17.211 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com'
2023-07-14T18:54:17.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-07-14T18:54:17.277 DEBUG:teuthology.orchestra.run.smithi104:> true
2023-07-14T18:54:17.405 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com'
2023-07-14T18:54:17.406 DEBUG:teuthology.orchestra.run.smithi104:> uname -m
2023-07-14T18:54:17.463 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64
2023-07-14T18:54:17.463 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release
2023-07-14T18:54:17.519 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Red Hat Enterprise Linux"
2023-07-14T18:54:17.520 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="8.6 (Ootpa)"
2023-07-14T18:54:17.520 INFO:teuthology.orchestra.run.smithi104.stdout:ID="rhel"
2023-07-14T18:54:17.520 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="fedora"
2023-07-14T18:54:17.520 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="8.6"
2023-07-14T18:54:17.521 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el8"
2023-07-14T18:54:17.521 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-07-14T18:54:17.521 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31"
2023-07-14T18:54:17.521 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-07-14T18:54:17.521 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.redhat.com/"
2023-07-14T18:54:17.521 INFO:teuthology.orchestra.run.smithi104.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-07-14T18:54:17.521 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-07-14T18:54:17.522 INFO:teuthology.orchestra.run.smithi104.stdout:
2023-07-14T18:54:17.522 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-07-14T18:54:17.522 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-07-14T18:54:17.522 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-07-14T18:54:17.522 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-07-14T18:54:17.523 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server
2023-07-14T18:54:22.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T18:54:22.551 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T18:54:27.664 INFO:teuthology.dispatcher.supervisor:Running job 7338051
2023-07-14T18:54:27.664 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_aa6a80e6cc655fff48452a7eca02070e61d074fc/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338051 --name pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi --description fs:workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/distributed export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/kernel_untar_build}} -- /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338051/orig.config.yaml
2023-07-14T18:54:27.669 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338051
2023-07-14T18:54:27.669 INFO:teuthology.dispatcher.supervisor:Job PID: 27625
2023-07-14T18:54:27.670 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-07-14T18:56:27.769 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T18:56:27.770 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T18:58:32.949 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T18:58:32.950 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:00:33.121 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:00:33.122 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:02:38.305 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:02:38.305 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:04:40.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:04:40.485 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:06:45.668 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:06:45.669 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:08:50.865 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:08:50.865 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:10:56.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:10:56.049 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:13:01.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:13:01.233 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:15:06.394 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:15:06.395 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:17:11.569 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:17:11.570 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:19:13.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:19:13.697 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:21:18.881 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:21:18.882 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:23:24.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:23:24.057 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:25:24.222 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:25:24.223 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:27:29.453 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:27:29.453 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:29:29.629 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-07-14T19:29:29.629 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-07-14T19:29:34.700 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-07-14T19:29:42.133 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-07-14T19:29:42.133 INFO:teuthology.nuke:Checking targets against current locks
2023-07-14T19:29:42.180 DEBUG:teuthology.nuke:shortname: smithi053
2023-07-14T19:29:42.180 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-14T19:29:42.203 DEBUG:teuthology.nuke:shortname: smithi070
2023-07-14T19:29:42.203 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-14T19:29:42.205 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338051', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:49:07.309276', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}
2023-07-14T19:29:42.221 DEBUG:teuthology.nuke:shortname: smithi104
2023-07-14T19:29:42.221 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-07-14T19:29:42.227 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338051', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:49:07.307930', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}
2023-07-14T19:29:42.247 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-07-14_16:44:47-fs:workload-wip-pdonnell-testing-20230714.134335-distro-default-smithi/7338051', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-14 18:49:07.306370', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}
2023-07-14T19:29:42.259 INFO:teuthology.orchestra.console:Power off smithi053
2023-07-14T19:29:42.259 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-14T19:29:42.282 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi053.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T19:29:42.300 INFO:teuthology.orchestra.console:Power off smithi070
2023-07-14T19:29:42.301 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-14T19:29:42.319 INFO:teuthology.orchestra.console:Power off smithi104
2023-07-14T19:29:42.319 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-07-14T19:29:42.331 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T19:29:42.343 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-07-14T19:29:42.430 INFO:teuthology.orchestra.console:Power off for smithi053 completed
2023-07-14T19:29:42.454 INFO:teuthology.orchestra.console:Power off for smithi070 completed
2023-07-14T19:29:42.466 INFO:teuthology.orchestra.console:Power off for smithi104 completed
2023-07-14T19:29:42.581 INFO:teuthology.lock.ops:unlocked: smithi053.front.sepia.ceph.com
2023-07-14T19:29:42.606 INFO:teuthology.lock.ops:unlocked: smithi070.front.sepia.ceph.com
2023-07-14T19:29:42.615 INFO:teuthology.lock.ops:unlocked: smithi104.front.sepia.ceph.com