2023-06-26T22:21:20.315 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb
2023-06-26T22:21:20.316 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:21:20.317 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:21:30.364 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f8a006d5670>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f8a018ee8b0>, signals=[15])
2023-06-26T22:21:30.365 INFO:teuthology.lock.ops:Start node 'smithi090.front.sepia.ceph.com' reimaging
2023-06-26T22:21:30.366 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: reset os type and version on server
2023-06-26T22:21:30.366 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2023-06-26T22:21:30.387 INFO:teuthology.lock.ops:Start node 'smithi188.front.sepia.ceph.com' reimaging
2023-06-26T22:21:30.387 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: reset os type and version on server
2023-06-26T22:21:30.388 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server
2023-06-26T22:21:30.389 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /host {"name": "smithi090"}
2023-06-26T22:21:40.418 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /host {"name": "smithi188"}
2023-06-26T22:21:50.277 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-06-26T22:22:05.326 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /image {"name": "smithi_rhel_8.4"}
2023-06-26T22:22:10.334 DEBUG:teuthology.provision.fog.smithi090:Requesting image smithi_rhel_8.4 (ID 7)
2023-06-26T22:22:10.335 DEBUG:teuthology.provision.fog.smithi090:FOG request: PUT /host/195 {"imageID": 7}
2023-06-26T22:22:15.396 DEBUG:teuthology.provision.fog.smithi188:Requesting image smithi_rhel_8.4 (ID 7)
2023-06-26T22:22:15.396 DEBUG:teuthology.provision.fog.smithi188:FOG request: PUT /host/293 {"imageID": 7}
2023-06-26T22:22:20.401 INFO:teuthology.provision.fog.smithi090:Scheduling deploy of rhel 8.4
2023-06-26T22:22:20.402 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:22:22.914 INFO:teuthology.provision.fog.smithi188:Scheduling deploy of rhel 8.4
2023-06-26T22:22:22.915 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:22:22.924 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /tasktype {"name": "deploy"}
2023-06-26T22:22:22.965 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /tasktype {"name": "deploy"}
2023-06-26T22:22:22.967 DEBUG:teuthology.provision.fog.smithi090:FOG request: POST /host/195/task {"taskTypeID": 1}
2023-06-26T22:22:23.000 DEBUG:teuthology.provision.fog.smithi188:FOG request: POST /host/293/task {"taskTypeID": 1}
2023-06-26T22:22:23.020 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:22:23.057 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:22:32.104 INFO:teuthology.orchestra.console:Power off smithi090
2023-06-26T22:22:32.105 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T22:22:32.121 INFO:teuthology.orchestra.console:Power off smithi188
2023-06-26T22:22:32.122 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T22:22:32.133 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:32.148 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:37.152 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:37.161 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:42.265 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:42.274 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:47.377 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:47.388 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:52.489 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:52.499 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:52.710 INFO:teuthology.orchestra.console:Power off for smithi188 completed
2023-06-26T22:22:52.811 INFO:teuthology.orchestra.console:Power on smithi188
2023-06-26T22:22:52.811 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-26T22:22:52.831 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:56.856 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:22:57.603 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:01.968 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:02.182 INFO:teuthology.orchestra.console:Power on for smithi188 completed
2023-06-26T22:23:02.283 INFO:teuthology.provision.fog.smithi188:Waiting for deploy to finish
2023-06-26T22:23:02.284 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:23:02.717 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:07.827 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:12.940 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:17.356 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:23:22.044 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:22.256 INFO:teuthology.orchestra.console:Power off for smithi090 completed
2023-06-26T22:23:22.357 INFO:teuthology.orchestra.console:Power on smithi090
2023-06-26T22:23:22.357 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-26T22:23:22.381 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:26.409 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:31.520 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:36.636 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:37.130 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:23:42.233 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:47.344 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:52.456 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:23:52.667 INFO:teuthology.orchestra.console:Power on for smithi090 completed
2023-06-26T22:23:52.768 INFO:teuthology.provision.fog.smithi090:Waiting for deploy to finish
2023-06-26T22:23:52.769 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:23:57.777 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:24:17.888 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:24:17.889 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:24:33.564 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:24:33.566 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:24:48.664 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:24:48.665 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:25:07.103 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:25:07.105 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:25:32.218 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:25:32.219 DEBUG:teuthology.provision.fog.smithi188:FOG request: GET /task/active None
2023-06-26T22:25:42.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:25:50.388 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2023-06-26T22:25:56.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:26:01.398 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:26:06.405 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2023-06-26T22:26:12.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:26:20.500 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2023-06-26T22:26:21.474 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:26:26.501 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:26:29.552 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2023-06-26T22:26:35.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:26:41.536 DEBUG:teuthology.provision.fog.smithi090:FOG request: GET /task/active None
2023-06-26T22:26:46.543 WARNING:teuthology.provision.fog.smithi188:[Errno None] Unable to connect to port 22 on 172.21.15.188
2023-06-26T22:26:46.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:26:52.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:26:57.548 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-06-26T22:27:03.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:27:11.632 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-06-26T22:27:17.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:27:20.724 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-06-26T22:27:26.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:27:32.516 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-26T22:27:32.517 DEBUG:teuthology.orchestra.run.smithi188:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-26T22:27:34.804 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-06-26T22:27:40.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:27:48.482 INFO:teuthology.provision.fog.smithi188:Node is ready
2023-06-26T22:27:48.482 DEBUG:teuthology.orchestra.run.smithi188:> hostname
2023-06-26T22:27:48.498 INFO:teuthology.orchestra.run.smithi188.stdout:smithi188
2023-06-26T22:27:48.499 DEBUG:teuthology.orchestra.run.smithi188:> grep smithi188 /etc/hosts
2023-06-26T22:27:48.553 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.15.188 smithi188.front.sepia.ceph.com smithi188
2023-06-26T22:27:48.553 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188
2023-06-26T22:27:48.696 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188/smithi188/g' /etc/hosts
2023-06-26T22:27:48.721 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/172.21.15.188/172.21.15.188/g' /etc/hosts
2023-06-26T22:27:48.786 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i -e 's/smithi188/smithi188/g' /etc/hostname
2023-06-26T22:27:48.850 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname smithi188
2023-06-26T22:27:48.880 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-06-26T22:27:48.915 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release
2023-06-26T22:27:48.969 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="Red Hat Enterprise Linux"
2023-06-26T22:27:48.969 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8.4 (Ootpa)"
2023-06-26T22:27:48.969 INFO:teuthology.orchestra.run.smithi188.stdout:ID="rhel"
2023-06-26T22:27:48.970 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="fedora"
2023-06-26T22:27:48.970 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8.4"
2023-06-26T22:27:48.970 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8"
2023-06-26T22:27:48.970 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-26T22:27:48.970 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31"
2023-06-26T22:27:48.970 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-26T22:27:48.970 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://www.redhat.com/"
2023-06-26T22:27:48.970 INFO:teuthology.orchestra.run.smithi188.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-26T22:27:48.971 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-26T22:27:48.971 INFO:teuthology.orchestra.run.smithi188.stdout:
2023-06-26T22:27:48.971 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-26T22:27:48.971 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-26T22:27:48.971 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-26T22:27:48.971 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-26T22:27:48.972 INFO:teuthology.provision.fog.smithi188:Deploy complete!
2023-06-26T22:27:48.974 DEBUG:teuthology.parallel:result is None
2023-06-26T22:27:54.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:28:18.416 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-06-26T22:28:24.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:28:27.476 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-06-26T22:28:33.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:28:41.556 WARNING:teuthology.provision.fog.smithi090:[Errno None] Unable to connect to port 22 on 172.21.15.90
2023-06-26T22:28:47.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:28:53.241 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-26T22:28:53.242 DEBUG:teuthology.orchestra.run.smithi090:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-26T22:29:03.511 INFO:teuthology.provision.fog.smithi090:Node is ready
2023-06-26T22:29:03.511 DEBUG:teuthology.orchestra.run.smithi090:> hostname
2023-06-26T22:29:03.569 INFO:teuthology.orchestra.run.smithi090.stdout:smithi036
2023-06-26T22:29:03.570 DEBUG:teuthology.orchestra.run.smithi090:> grep smithi036 /etc/hosts
2023-06-26T22:29:03.626 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.36 smithi036.front.sepia.ceph.com smithi036
2023-06-26T22:29:03.626 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090
2023-06-26T22:29:03.785 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi036/smithi090/g' /etc/hosts
2023-06-26T22:29:03.813 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/172.21.15.36/172.21.15.90/g' /etc/hosts
2023-06-26T22:29:03.879 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i -e 's/smithi036/smithi090/g' /etc/hostname
2023-06-26T22:29:03.945 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname smithi090
2023-06-26T22:29:04.010 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release
2023-06-26T22:29:04.080 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Red Hat Enterprise Linux"
2023-06-26T22:29:04.080 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="8.4 (Ootpa)"
2023-06-26T22:29:04.080 INFO:teuthology.orchestra.run.smithi090.stdout:ID="rhel"
2023-06-26T22:29:04.080 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="fedora"
2023-06-26T22:29:04.081 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="8.4"
2023-06-26T22:29:04.081 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el8"
2023-06-26T22:29:04.081 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-26T22:29:04.081 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2023-06-26T22:29:04.081 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-26T22:29:04.082 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.redhat.com/"
2023-06-26T22:29:04.082 INFO:teuthology.orchestra.run.smithi090.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-26T22:29:04.082 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-26T22:29:04.082 INFO:teuthology.orchestra.run.smithi090.stdout:
2023-06-26T22:29:04.083 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-26T22:29:04.083 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-26T22:29:04.083 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-26T22:29:04.083 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-26T22:29:04.084 INFO:teuthology.provision.fog.smithi090:Deploy complete!
2023-06-26T22:29:04.084 DEBUG:teuthology.parallel:result is None
2023-06-26T22:29:09.280 INFO:teuthology.lock.ops:Checking smithi090.front.sepia.ceph.com
2023-06-26T22:29:09.282 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-26T22:29:14.309 INFO:teuthology.lock.ops:Checking smithi188.front.sepia.ceph.com
2023-06-26T22:29:14.310 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-26T22:29:19.343 INFO:teuthology.lock.ops:Updating [smithi090.front.sepia.ceph.com]: set os type and version on server
2023-06-26T22:29:19.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com'
2023-06-26T22:29:19.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:29:24.421 DEBUG:teuthology.orchestra.run.smithi090:> true
2023-06-26T22:29:24.539 DEBUG:teuthology.orchestra.run.smithi090:> uname -m
2023-06-26T22:29:24.594 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64
2023-06-26T22:29:24.594 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release
2023-06-26T22:29:24.649 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Red Hat Enterprise Linux"
2023-06-26T22:29:24.649 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="8.4 (Ootpa)"
2023-06-26T22:29:24.649 INFO:teuthology.orchestra.run.smithi090.stdout:ID="rhel"
2023-06-26T22:29:24.649 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="fedora"
2023-06-26T22:29:24.649 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="8.4"
2023-06-26T22:29:24.650 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el8"
2023-06-26T22:29:24.650 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-26T22:29:24.650 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31"
2023-06-26T22:29:24.650 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-26T22:29:24.650 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.redhat.com/"
2023-06-26T22:29:24.650 INFO:teuthology.orchestra.run.smithi090.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-26T22:29:24.650 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-26T22:29:24.651 INFO:teuthology.orchestra.run.smithi090.stdout:
2023-06-26T22:29:24.651 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-26T22:29:24.651 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-26T22:29:24.651 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-26T22:29:24.651 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-26T22:29:24.652 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server
2023-06-26T22:29:29.676 INFO:teuthology.lock.ops:Updating [smithi188.front.sepia.ceph.com]: set os type and version on server
2023-06-26T22:29:29.676 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com'
2023-06-26T22:29:29.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-26T22:29:34.749 DEBUG:teuthology.orchestra.run.smithi188:> true
2023-06-26T22:29:34.871 DEBUG:teuthology.orchestra.run.smithi188:> uname -m
2023-06-26T22:29:34.927 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64
2023-06-26T22:29:34.927 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release
2023-06-26T22:29:34.981 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="Red Hat Enterprise Linux"
2023-06-26T22:29:34.982 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="8.4 (Ootpa)"
2023-06-26T22:29:34.982 INFO:teuthology.orchestra.run.smithi188.stdout:ID="rhel"
2023-06-26T22:29:34.982 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="fedora"
2023-06-26T22:29:34.982 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="8.4"
2023-06-26T22:29:34.982 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el8"
2023-06-26T22:29:34.983 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
2023-06-26T22:29:34.983 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31"
2023-06-26T22:29:34.983 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
2023-06-26T22:29:34.983 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://www.redhat.com/"
2023-06-26T22:29:34.983 INFO:teuthology.orchestra.run.smithi188.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/"
2023-06-26T22:29:34.983 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-26T22:29:34.983 INFO:teuthology.orchestra.run.smithi188.stdout:
2023-06-26T22:29:34.984 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-26T22:29:34.984 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
2023-06-26T22:29:34.984 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
2023-06-26T22:29:34.984 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4"
2023-06-26T22:29:34.985 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server
2023-06-26T22:29:40.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:29:40.016 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:29:45.121 INFO:teuthology.dispatcher.supervisor:Running job 7317023
2023-06-26T22:29:45.122 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-06-26_19:12:21-fs-wip-yuri4-testing-2023-06-26-0711-quincy-distro-default-smithi/7317023 --name yuriw-2023-06-26_19:12:21-fs-wip-yuri4-testing-2023-06-26-0711-quincy-distro-default-smithi --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} -- /home/teuthworker/archive/yuriw-2023-06-26_19:12:21-fs-wip-yuri4-testing-2023-06-26-0711-quincy-distro-default-smithi/7317023/orig.config.yaml
2023-06-26T22:29:45.125 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-06-26_19:12:21-fs-wip-yuri4-testing-2023-06-26-0711-quincy-distro-default-smithi/7317023
2023-06-26T22:29:45.126 INFO:teuthology.dispatcher.supervisor:Job PID: 7947
2023-06-26T22:29:45.126 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2023-06-26T22:31:45.219 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:31:45.221 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:33:48.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:33:48.967 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:35:54.149 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:35:54.150 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:37:59.331 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:37:59.332 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:40:04.509 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:40:04.509 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:42:09.305 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:42:09.306 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:44:13.585 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:44:13.586 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:46:18.773 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:46:18.773 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:48:23.949 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:48:23.951 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:50:29.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:50:29.133 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:52:34.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:52:34.321 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:54:39.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-26T22:54:39.506 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-26T22:54:44.588 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120
2023-06-26T22:54:54.781 INFO:teuthology.dispatcher.supervisor:Nuking machines...
2023-06-26T22:54:54.781 INFO:teuthology.nuke:Checking targets against current locks
2023-06-26T22:55:13.248 DEBUG:teuthology.nuke:shortname: smithi090
2023-06-26T22:55:13.249 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-26T22:55:13.277 DEBUG:teuthology.nuke:shortname: smithi188
2023-06-26T22:55:13.277 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-26T22:55:13.280 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-26_19:12:21-fs-wip-yuri4-testing-2023-06-26-0711-quincy-distro-default-smithi/7317023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-26 22:21:16.212073', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFA1cU4W3bGDK81tulUj3Yz9hLvYM9nyebriiRUar84MO0RAwdFqmtM+sIGnvjy6eq3G3RenZIRamAvLj0nyUnE='}
2023-06-26T22:55:13.299 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-26_19:12:21-fs-wip-yuri4-testing-2023-06-26-0711-quincy-distro-default-smithi/7317023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-26 22:21:16.210004', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGlwgvneI2xDREhSHluZ2Ku2cN1Omr+t5HwYzod6ZMCXIaOSqGfrN+3c4rtsGauLvBRF6UodCuPSwSiTI4QkwOg='}
2023-06-26T22:55:13.337 INFO:teuthology.orchestra.console:Power off smithi090
2023-06-26T22:55:13.338 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T22:55:13.358 INFO:teuthology.orchestra.console:Power off smithi188
2023-06-26T22:55:13.359 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-26T22:55:13.370 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:18.384 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:23.496 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:28.394 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:28.607 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:33.408 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:33.719 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:38.524 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:38.830 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:39.044 INFO:teuthology.orchestra.console:Power off for smithi090 completed
2023-06-26T22:55:43.762 INFO:teuthology.lock.ops:unlocked: smithi090.front.sepia.ceph.com
2023-06-26T22:55:47.708 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:52.820 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:55:57.932 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:56:03.044 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:56:08.156 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:56:13.268 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-26T22:56:13.478 INFO:teuthology.orchestra.console:Power off for smithi188 completed
2023-06-26T22:56:23.681 INFO:teuthology.lock.ops:unlocked: smithi188.front.sepia.ceph.com