2023-06-21T07:19:31.616 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb
2023-06-21T07:19:31.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:19:31.618 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:19:31.812 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fa40e230700>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fa40dba8670>, signals=[15])
2023-06-21T07:19:31.815 INFO:teuthology.lock.ops:Start node 'smithi073.front.sepia.ceph.com' reimaging
2023-06-21T07:19:31.815 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: reset os type and version on server
2023-06-21T07:19:31.816 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server
2023-06-21T07:19:31.840 INFO:teuthology.lock.ops:Start node 'smithi151.front.sepia.ceph.com' reimaging
2023-06-21T07:19:31.840 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: reset os type and version on server
2023-06-21T07:19:31.840 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server
2023-06-21T07:19:31.842 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /host {"name": "smithi073"}
2023-06-21T07:19:31.864 INFO:teuthology.lock.ops:Start node 'smithi187.front.sepia.ceph.com' reimaging
2023-06-21T07:19:31.865 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: reset os type and version on server
2023-06-21T07:19:31.865 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server
2023-06-21T07:19:31.866 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /host {"name": "smithi151"}
2023-06-21T07:19:31.892 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /host {"name": "smithi187"}
2023-06-21T07:19:31.922 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-06-21T07:19:31.934 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-06-21T07:19:31.961 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-06-21T07:19:31.964 DEBUG:teuthology.provision.fog.smithi073:Requesting image smithi_rhel_8.6 (ID 5)
2023-06-21T07:19:31.964 DEBUG:teuthology.provision.fog.smithi073:FOG request: PUT /host/178 {"imageID": 5}
2023-06-21T07:19:31.973 DEBUG:teuthology.provision.fog.smithi151:Requesting image smithi_rhel_8.6 (ID 5)
2023-06-21T07:19:31.973 DEBUG:teuthology.provision.fog.smithi151:FOG request: PUT /host/256 {"imageID": 5}
2023-06-21T07:19:32.002 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of rhel 8.6
2023-06-21T07:19:32.002 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:19:32.005 DEBUG:teuthology.provision.fog.smithi187:Requesting image smithi_rhel_8.6 (ID 5)
2023-06-21T07:19:32.005 DEBUG:teuthology.provision.fog.smithi187:FOG request: PUT /host/292 {"imageID": 5}
2023-06-21T07:19:32.018 INFO:teuthology.provision.fog.smithi151:Scheduling deploy of rhel 8.6
2023-06-21T07:19:32.019 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:19:32.046 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of rhel 8.6
2023-06-21T07:19:32.046 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:19:32.062 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /tasktype {"name": "deploy"}
2023-06-21T07:19:32.069 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /tasktype {"name": "deploy"}
2023-06-21T07:19:32.096 DEBUG:teuthology.provision.fog.smithi073:FOG request: POST /host/178/task {"taskTypeID": 1}
2023-06-21T07:19:32.098 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /tasktype {"name": "deploy"}
2023-06-21T07:19:32.101 DEBUG:teuthology.provision.fog.smithi151:FOG request: POST /host/256/task {"taskTypeID": 1}
2023-06-21T07:19:32.133 DEBUG:teuthology.provision.fog.smithi187:FOG request: POST /host/292/task {"taskTypeID": 1}
2023-06-21T07:19:32.147 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:19:32.150 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:19:32.193 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:19:32.250 INFO:teuthology.orchestra.console:Power off smithi073
2023-06-21T07:19:32.251 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-21T07:19:32.270 INFO:teuthology.orchestra.console:Power off smithi151
2023-06-21T07:19:32.271 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-21T07:19:32.282 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:32.292 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:32.312 INFO:teuthology.orchestra.console:Power off smithi187
2023-06-21T07:19:32.313 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-21T07:19:32.336 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:32.405 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2023-06-21T07:19:32.457 INFO:teuthology.orchestra.console:Power off for smithi187 completed
2023-06-21T07:19:32.506 INFO:teuthology.orchestra.console:Power on smithi073
2023-06-21T07:19:32.507 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-21T07:19:32.528 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:32.557 INFO:teuthology.orchestra.console:Power on smithi187
2023-06-21T07:19:32.558 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-21T07:19:32.580 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:36.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:36.552 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:36.603 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:36.763 INFO:teuthology.orchestra.console:Power on for smithi073 completed
2023-06-21T07:19:36.865 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish
2023-06-21T07:19:36.865 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:19:40.440 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:40.650 INFO:teuthology.orchestra.console:Power off for smithi151 completed
2023-06-21T07:19:40.713 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:40.752 INFO:teuthology.orchestra.console:Power on smithi151
2023-06-21T07:19:40.752 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-21T07:19:40.774 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:40.926 INFO:teuthology.orchestra.console:Power on for smithi187 completed
2023-06-21T07:19:41.030 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish
2023-06-21T07:19:41.031 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:19:44.796 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:48.908 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:19:49.119 INFO:teuthology.orchestra.console:Power on for smithi151 completed
2023-06-21T07:19:49.221 INFO:teuthology.provision.fog.smithi151:Waiting for deploy to finish
2023-06-21T07:19:49.221 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:19:51.932 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:19:56.092 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:20:04.299 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:20:06.998 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:20:11.164 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:20:19.384 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:20:22.080 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:20:26.225 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:20:34.484 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:20:37.180 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:20:41.307 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:20:49.569 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:20:52.272 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:20:56.400 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:21:04.648 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:21:07.360 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:21:11.496 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:21:19.732 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:21:22.440 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:21:26.574 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:21:34.808 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:21:37.504 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:21:41.644 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:21:49.881 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:21:52.580 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:21:56.720 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:22:04.970 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:22:07.663 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:22:11.804 DEBUG:teuthology.provision.fog.smithi187:FOG request: GET /task/active None
2023-06-21T07:22:11.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:22:11.957 ERROR:teuthology.orchestra.connection:Error authenticating with smithi187.front.sepia.ceph.com: Authentication failed.
2023-06-21T07:22:20.043 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:22:22.737 DEBUG:teuthology.provision.fog.smithi073:FOG request: GET /task/active None
2023-06-21T07:22:22.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:22:25.872 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-06-21T07:22:31.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:22:34.928 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-06-21T07:22:35.113 DEBUG:teuthology.provision.fog.smithi151:FOG request: GET /task/active None
2023-06-21T07:22:35.182 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:22:38.256 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:22:40.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:22:43.988 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-06-21T07:22:44.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:22:47.316 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:22:50.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:22:53.232 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-06-21T07:22:53.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:22:56.368 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:22:59.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:02.288 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-06-21T07:23:02.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:05.424 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:23:08.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:11.344 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73
2023-06-21T07:23:11.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:12.961 WARNING:teuthology.provision.fog.smithi187:timed out
2023-06-21T07:23:14.480 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:23:17.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:18.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:20.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:23.536 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:23:29.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:32.596 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:23:38.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:41.648 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:23:47.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:23:50.845 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-21T07:23:50.846 DEBUG:teuthology.orchestra.run.smithi187:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-21T07:23:51.134 INFO:teuthology.provision.fog.smithi187:Node is ready
2023-06-21T07:23:51.135 DEBUG:teuthology.orchestra.run.smithi187:> hostname
2023-06-21T07:23:51.150 INFO:teuthology.orchestra.run.smithi187.stdout:smithi022
2023-06-21T07:23:51.151 DEBUG:teuthology.orchestra.run.smithi187:> grep smithi022 /etc/hosts
2023-06-21T07:23:51.208 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-06-21T07:23:51.209 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname smithi187
2023-06-21T07:23:51.337 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/smithi022/smithi187/g' /etc/hosts
2023-06-21T07:23:51.364 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/172.21.15.22/172.21.15.187/g' /etc/hosts
2023-06-21T07:23:51.433 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/smithi022/smithi187/g' /etc/hostname
2023-06-21T07:23:51.500 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname smithi187
2023-06-21T07:23:51.569 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release
2023-06-21T07:23:51.627 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux"
2023-06-21T07:23:51.627 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.6 (Ootpa)"
2023-06-21T07:23:51.627 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel"
2023-06-21T07:23:51.628 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora"
2023-06-21T07:23:51.628 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.6"
2023-06-21T07:23:51.628 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8"
2023-06-21T07:23:51.628 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-06-21T07:23:51.628 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31"
2023-06-21T07:23:51.629 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-06-21T07:23:51.629 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/"
2023-06-21T07:23:51.629 INFO:teuthology.orchestra.run.smithi187.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-21T07:23:51.629 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-21T07:23:51.630 INFO:teuthology.orchestra.run.smithi187.stdout:
2023-06-21T07:23:51.630 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-21T07:23:51.630 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-06-21T07:23:51.630 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-21T07:23:51.631 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-06-21T07:23:51.632 INFO:teuthology.provision.fog.smithi187:Deploy complete!
2023-06-21T07:23:51.633 DEBUG:teuthology.parallel:result is None
2023-06-21T07:23:57.936 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:24:03.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:24:06.992 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:24:12.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:24:16.048 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151
2023-06-21T07:24:17.347 WARNING:teuthology.provision.fog.smithi073:timed out
2023-06-21T07:24:22.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:24:23.150 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-21T07:24:23.150 DEBUG:teuthology.orchestra.run.smithi151:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-21T07:24:23.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:24:23.440 INFO:teuthology.provision.fog.smithi151:Node is ready
2023-06-21T07:24:23.440 DEBUG:teuthology.orchestra.run.smithi151:> hostname
2023-06-21T07:24:23.456 INFO:teuthology.orchestra.run.smithi151.stdout:smithi022
2023-06-21T07:24:23.456 DEBUG:teuthology.orchestra.run.smithi151:> grep smithi022 /etc/hosts
2023-06-21T07:24:23.512 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022
2023-06-21T07:24:23.512 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151
2023-06-21T07:24:23.591 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-21T07:24:23.592 DEBUG:teuthology.orchestra.run.smithi073:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-21T07:24:23.627 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi022/smithi151/g' /etc/hosts
2023-06-21T07:24:23.654 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/172.21.15.22/172.21.15.151/g' /etc/hosts
2023-06-21T07:24:23.719 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi022/smithi151/g' /etc/hostname
2023-06-21T07:24:23.786 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151
2023-06-21T07:24:23.851 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release
2023-06-21T07:24:23.908 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="Red Hat Enterprise Linux"
2023-06-21T07:24:23.908 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8.6 (Ootpa)"
2023-06-21T07:24:23.908 INFO:teuthology.orchestra.run.smithi151.stdout:ID="rhel"
2023-06-21T07:24:23.908 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="fedora"
2023-06-21T07:24:23.908 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8.6"
2023-06-21T07:24:23.909 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8"
2023-06-21T07:24:23.909 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-06-21T07:24:23.909 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31"
2023-06-21T07:24:23.909 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-06-21T07:24:23.909 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://www.redhat.com/"
2023-06-21T07:24:23.909 INFO:teuthology.orchestra.run.smithi151.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-21T07:24:23.909 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-21T07:24:23.910 INFO:teuthology.orchestra.run.smithi151.stdout:
2023-06-21T07:24:23.910 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-21T07:24:23.910 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-06-21T07:24:23.910 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-21T07:24:23.910 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-06-21T07:24:23.910 INFO:teuthology.provision.fog.smithi151:Deploy complete!
2023-06-21T07:24:23.911 DEBUG:teuthology.parallel:result is None
2023-06-21T07:24:23.912 INFO:teuthology.provision.fog.smithi073:Node is ready
2023-06-21T07:24:23.912 DEBUG:teuthology.orchestra.run.smithi073:> hostname
2023-06-21T07:24:23.929 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073
2023-06-21T07:24:23.929 DEBUG:teuthology.orchestra.run.smithi073:> grep smithi073 /etc/hosts
2023-06-21T07:24:23.987 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73 smithi073.front.sepia.ceph.com smithi073
2023-06-21T07:24:23.988 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname smithi073
2023-06-21T07:24:24.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/smithi073/smithi073/g' /etc/hosts
2023-06-21T07:24:24.138 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/172.21.15.73/172.21.15.73/g' /etc/hosts
2023-06-21T07:24:24.205 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/smithi073/smithi073/g' /etc/hostname
2023-06-21T07:24:24.272 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname smithi073
2023-06-21T07:24:24.338 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release
2023-06-21T07:24:24.393 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Red Hat Enterprise Linux"
2023-06-21T07:24:24.394 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8.6 (Ootpa)"
2023-06-21T07:24:24.394 INFO:teuthology.orchestra.run.smithi073.stdout:ID="rhel"
2023-06-21T07:24:24.394 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="fedora"
2023-06-21T07:24:24.394 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8.6"
2023-06-21T07:24:24.394 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8"
2023-06-21T07:24:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-06-21T07:24:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31"
2023-06-21T07:24:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-06-21T07:24:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.redhat.com/"
2023-06-21T07:24:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-21T07:24:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-21T07:24:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:
2023-06-21T07:24:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-21T07:24:24.396 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-06-21T07:24:24.396 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-21T07:24:24.396 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-06-21T07:24:24.396 INFO:teuthology.provision.fog.smithi073:Deploy complete!
2023-06-21T07:24:24.397 DEBUG:teuthology.parallel:result is None
2023-06-21T07:24:24.585 INFO:teuthology.lock.ops:Checking smithi073.front.sepia.ceph.com
2023-06-21T07:24:24.586 INFO:teuthology.lock.ops:Checking smithi151.front.sepia.ceph.com
2023-06-21T07:24:24.586 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-21T07:24:24.612 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com
2023-06-21T07:24:24.614 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: set os type and version on server
2023-06-21T07:24:24.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com'
2023-06-21T07:24:24.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:24:24.681 DEBUG:teuthology.orchestra.run.smithi073:> true
2023-06-21T07:24:24.703 DEBUG:teuthology.orchestra.run.smithi073:> uname -m
2023-06-21T07:24:24.760 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64
2023-06-21T07:24:24.761 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release
2023-06-21T07:24:24.816 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Red Hat Enterprise Linux"
2023-06-21T07:24:24.817 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8.6 (Ootpa)"
2023-06-21T07:24:24.817 INFO:teuthology.orchestra.run.smithi073.stdout:ID="rhel"
2023-06-21T07:24:24.817 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="fedora"
2023-06-21T07:24:24.817 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8.6"
2023-06-21T07:24:24.817 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8"
2023-06-21T07:24:24.818 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-06-21T07:24:24.818 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31"
2023-06-21T07:24:24.818 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-06-21T07:24:24.818 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.redhat.com/"
2023-06-21T07:24:24.818 INFO:teuthology.orchestra.run.smithi073.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-21T07:24:24.818 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-21T07:24:24.818 INFO:teuthology.orchestra.run.smithi073.stdout:
2023-06-21T07:24:24.819 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-21T07:24:24.819 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-06-21T07:24:24.819 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-21T07:24:24.819 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-06-21T07:24:24.819 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server
2023-06-21T07:24:24.845 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: set os type and version on server
2023-06-21T07:24:24.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com'
2023-06-21T07:24:24.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:24:24.910 DEBUG:teuthology.orchestra.run.smithi151:> true
2023-06-21T07:24:24.978 DEBUG:teuthology.orchestra.run.smithi151:> uname -m
2023-06-21T07:24:25.034 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64
2023-06-21T07:24:25.035 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release
2023-06-21T07:24:25.092 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="Red Hat Enterprise Linux"
2023-06-21T07:24:25.092 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8.6 (Ootpa)"
2023-06-21T07:24:25.093 INFO:teuthology.orchestra.run.smithi151.stdout:ID="rhel"
2023-06-21T07:24:25.093 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="fedora"
2023-06-21T07:24:25.093 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8.6"
2023-06-21T07:24:25.094 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8"
2023-06-21T07:24:25.094 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-06-21T07:24:25.094 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31"
2023-06-21T07:24:25.094 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-06-21T07:24:25.095 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://www.redhat.com/"
2023-06-21T07:24:25.095 INFO:teuthology.orchestra.run.smithi151.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-21T07:24:25.095 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-21T07:24:25.095 INFO:teuthology.orchestra.run.smithi151.stdout:
2023-06-21T07:24:25.095 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-21T07:24:25.095 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-06-21T07:24:25.095 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-21T07:24:25.095 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-06-21T07:24:25.096 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server
2023-06-21T07:24:25.120 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server
2023-06-21T07:24:25.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com'
2023-06-21T07:24:25.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-21T07:24:25.186 DEBUG:teuthology.orchestra.run.smithi187:> true
2023-06-21T07:24:25.318 DEBUG:teuthology.orchestra.run.smithi187:> uname -m
2023-06-21T07:24:25.376 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64
2023-06-21T07:24:25.377 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release
2023-06-21T07:24:25.434 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux"
2023-06-21T07:24:25.435 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.6 (Ootpa)"
2023-06-21T07:24:25.435 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel"
2023-06-21T07:24:25.435 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora"
2023-06-21T07:24:25.436 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.6"
2023-06-21T07:24:25.436 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8"
2023-06-21T07:24:25.436 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)"
2023-06-21T07:24:25.437 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31"
2023-06-21T07:24:25.437 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos"
2023-06-21T07:24:25.437 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/"
2023-06-21T07:24:25.437 INFO:teuthology.orchestra.run.smithi187.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-21T07:24:25.438 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-21T07:24:25.438 INFO:teuthology.orchestra.run.smithi187.stdout:
2023-06-21T07:24:25.438 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-21T07:24:25.438 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6
2023-06-21T07:24:25.439 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-21T07:24:25.439 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6"
2023-06-21T07:24:25.440 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server
2023-06-21T07:24:25.463 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:24:25.463 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:24:25.566 INFO:teuthology.dispatcher.supervisor:Running job 7309516
2023-06-21T07:24:25.566 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-06-21_05:44:47-fs-wip-vshankar-testing-20230620.131636-testing-default-smithi/7309516 --name vshankar-2023-06-21_05:44:47-fs-wip-vshankar-testing-20230620.131636-testing-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/pjd}} -- /home/teuthworker/archive/vshankar-2023-06-21_05:44:47-fs-wip-vshankar-testing-20230620.131636-testing-default-smithi/7309516/orig.config.yaml
2023-06-21T07:24:25.570 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-06-21_05:44:47-fs-wip-vshankar-testing-20230620.131636-testing-default-smithi/7309516
2023-06-21T07:24:25.571 INFO:teuthology.dispatcher.supervisor:Job PID: 25000
2023-06-21T07:24:25.571 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-06-21T07:26:25.572 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:26:25.574 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:28:25.666 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:28:25.667 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:30:25.740 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:30:25.741 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:32:25.811 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:32:25.812 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:34:25.886 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:34:25.887 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:36:25.960 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:36:25.962 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:38:26.040 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:38:26.040 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:40:26.106 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:40:26.107 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:42:26.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:42:26.184 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:44:26.257 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:44:26.258 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:46:26.330 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:46:26.357 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:48:26.446 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:48:26.447 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:50:26.518 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-21T07:50:26.518 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-21T07:50:26.610 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-06-21T07:50:26.816 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-06-21T07:50:26.816 INFO:teuthology.nuke:Checking targets against current locks
2023-06-21T07:50:26.868 DEBUG:teuthology.nuke:shortname: smithi073
2023-06-21T07:50:26.868 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-21T07:50:26.892 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-21_05:44:47-fs-wip-vshankar-testing-20230620.131636-testing-default-smithi/7309516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-21 07:19:30.770354', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}
2023-06-21T07:50:26.895 DEBUG:teuthology.nuke:shortname: smithi151
2023-06-21T07:50:26.895 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-21T07:50:26.914 DEBUG:teuthology.nuke:shortname: smithi187
2023-06-21T07:50:26.914 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-21T07:50:26.920 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-21_05:44:47-fs-wip-vshankar-testing-20230620.131636-testing-default-smithi/7309516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-21 07:19:30.768170', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}
2023-06-21T07:50:26.947 INFO:teuthology.orchestra.console:Power off smithi073
2023-06-21T07:50:26.948 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-21T07:50:26.959 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-21_05:44:47-fs-wip-vshankar-testing-20230620.131636-testing-default-smithi/7309516', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-21 07:19:30.772264', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}
2023-06-21T07:50:26.970 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:50:26.987 INFO:teuthology.orchestra.console:Power off smithi151
2023-06-21T07:50:26.988 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-21T07:50:27.010 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:50:27.032 INFO:teuthology.orchestra.console:Power off smithi187
2023-06-21T07:50:27.033 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-21T07:50:27.054 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-21T07:50:27.102 INFO:teuthology.orchestra.console:Power off for smithi073 completed
2023-06-21T07:50:27.144 INFO:teuthology.orchestra.console:Power off for smithi151 completed
2023-06-21T07:50:27.177 INFO:teuthology.orchestra.console:Power off for smithi187 completed
2023-06-21T07:50:27.286 INFO:teuthology.lock.ops:unlocked: smithi073.front.sepia.ceph.com
2023-06-21T07:50:27.300 INFO:teuthology.lock.ops:unlocked: smithi151.front.sepia.ceph.com
2023-06-21T07:50:27.332 INFO:teuthology.lock.ops:unlocked: smithi187.front.sepia.ceph.com