2023-06-24T14:46:29.713 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb
2023-06-24T14:46:29.714 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-24T14:46:29.715 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-24T14:46:39.925 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f5242a59610>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f52423b44c0>, signals=[15])
2023-06-24T14:46:39.927 INFO:teuthology.lock.ops:Start node 'smithi084.front.sepia.ceph.com' reimaging
2023-06-24T14:46:39.928 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: reset os type and version on server
2023-06-24T14:46:39.928 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2023-06-24T14:46:39.950 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging
2023-06-24T14:46:39.951 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server
2023-06-24T14:46:39.951 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2023-06-24T14:46:39.953 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /host {"name": "smithi084"}
2023-06-24T14:46:44.982 INFO:teuthology.lock.ops:Start node 'smithi161.front.sepia.ceph.com' reimaging
2023-06-24T14:46:44.983 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: reset os type and version on server
2023-06-24T14:46:44.983 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server
2023-06-24T14:46:44.985 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /host {"name": "smithi154"}
2023-06-24T14:47:00.003 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /host {"name": "smithi161"}
2023-06-24T14:47:10.018 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-24T14:47:16.493 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-24T14:47:21.502 DEBUG:teuthology.provision.fog.smithi084:Requesting image smithi_centos_8.stream (ID 3)
2023-06-24T14:47:21.502 DEBUG:teuthology.provision.fog.smithi084:FOG request: PUT /host/189 {"imageID": 3}
2023-06-24T14:47:26.510 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-24T14:47:30.498 DEBUG:teuthology.provision.fog.smithi154:Requesting image smithi_centos_8.stream (ID 3)
2023-06-24T14:47:30.499 DEBUG:teuthology.provision.fog.smithi154:FOG request: PUT /host/259 {"imageID": 3}
2023-06-24T14:47:30.552 DEBUG:teuthology.provision.fog.smithi161:Requesting image smithi_centos_8.stream (ID 3)
2023-06-24T14:47:30.552 DEBUG:teuthology.provision.fog.smithi161:FOG request: PUT /host/266 {"imageID": 3}
2023-06-24T14:47:30.554 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of centos 8.stream
2023-06-24T14:47:30.554 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:47:30.558 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of centos 8.stream
2023-06-24T14:47:30.559 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-24T14:47:30.608 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of centos 8.stream
2023-06-24T14:47:30.609 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /task/active None
2023-06-24T14:47:30.619 DEBUG:teuthology.provision.fog.smithi084:Canceling deploy task with ID 342539
2023-06-24T14:47:30.620 DEBUG:teuthology.provision.fog.smithi084:FOG request: DELETE /task/cancel {"id": 342539}
2023-06-24T14:47:30.632 DEBUG:teuthology.provision.fog.smithi154:Canceling deploy task with ID 342536
2023-06-24T14:47:30.632 DEBUG:teuthology.provision.fog.smithi154:FOG request: DELETE /task/cancel {"id": 342536}
2023-06-24T14:47:30.675 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /tasktype {"name": "deploy"}
2023-06-24T14:47:30.681 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /tasktype {"name": "deploy"}
2023-06-24T14:47:30.687 DEBUG:teuthology.provision.fog.smithi161:Canceling deploy task with ID 342538
2023-06-24T14:47:30.687 DEBUG:teuthology.provision.fog.smithi161:FOG request: DELETE /task/cancel {"id": 342538}
2023-06-24T14:47:30.717 DEBUG:teuthology.provision.fog.smithi084:FOG request: POST /host/189/task {"taskTypeID": 1}
2023-06-24T14:47:30.723 DEBUG:teuthology.provision.fog.smithi154:FOG request: POST /host/259/task {"taskTypeID": 1}
2023-06-24T14:47:30.739 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /tasktype {"name": "deploy"}
2023-06-24T14:47:30.767 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:47:30.784 DEBUG:teuthology.provision.fog.smithi161:FOG request: POST /host/266/task {"taskTypeID": 1}
2023-06-24T14:47:30.787 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-24T14:47:35.841 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /task/active None
2023-06-24T14:47:41.164 INFO:teuthology.orchestra.console:Power off smithi154
2023-06-24T14:47:41.165 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-24T14:47:41.180 INFO:teuthology.orchestra.console:Power off smithi084
2023-06-24T14:47:41.181 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-24T14:47:46.209 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:47:46.218 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:47:46.240 INFO:teuthology.orchestra.console:Power off smithi161
2023-06-24T14:47:46.241 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-24T14:47:46.260 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:47:46.350 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2023-06-24T14:47:46.451 INFO:teuthology.orchestra.console:Power on smithi154
2023-06-24T14:47:46.452 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-24T14:47:46.473 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:47:50.284 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:47:50.494 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:47:50.705 INFO:teuthology.orchestra.console:Power on for smithi154 completed
2023-06-24T14:47:50.806 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish
2023-06-24T14:47:50.807 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-24T14:47:55.814 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:47:55.825 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:47:55.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:47:59.099 INFO:teuthology.orchestra.console:Power off for smithi161 completed
2023-06-24T14:47:59.200 INFO:teuthology.orchestra.console:Power on smithi161
2023-06-24T14:47:59.200 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-24T14:47:59.222 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:02.032 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:48:03.244 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:04.000 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:07.356 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:07.566 INFO:teuthology.orchestra.console:Power on for smithi161 completed
2023-06-24T14:48:07.667 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish
2023-06-24T14:48:07.668 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /task/active None
2023-06-24T14:48:10.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:48:10.491 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:10.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:48:15.574 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:48:15.675 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2023-06-24T14:48:15.776 INFO:teuthology.orchestra.console:Power on smithi084
2023-06-24T14:48:15.777 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-24T14:48:18.644 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:48:20.798 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:21.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:48:26.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:48:31.584 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:48:34.640 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:48:35.586 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:37.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:48:42.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:48:47.596 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:47.605 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:48:50.672 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:48:52.707 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:48:52.917 INFO:teuthology.orchestra.console:Power on for smithi084 completed
2023-06-24T14:48:53.018 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish
2023-06-24T14:48:53.019 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:48:58.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:49:03.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:49:06.096 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:49:06.100 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:49:12.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:49:12.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:49:15.152 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:49:18.099 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:49:23.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:49:28.112 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:49:34.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:49:34.579 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:49:40.586 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:49:45.593 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:49:50.598 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:49:50.672 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:49:56.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:50:01.756 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:01.756 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:02.831 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:02.873 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:13.841 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:50:18.917 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:18.918 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:29.997 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:29.997 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:32.610 ERROR:paramiko.transport:Exception (client): Error reading SSH protocol banner
2023-06-24T14:50:32.611 ERROR:paramiko.transport:Traceback (most recent call last):
2023-06-24T14:50:32.612 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2270, in _check_banner
2023-06-24T14:50:32.612 ERROR:paramiko.transport:    buf = self.packetizer.readline(timeout)
2023-06-24T14:50:32.612 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 380, in readline
2023-06-24T14:50:32.613 ERROR:paramiko.transport:    buf += self._read_timeout(timeout)
2023-06-24T14:50:32.613 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 609, in _read_timeout
2023-06-24T14:50:32.613 ERROR:paramiko.transport:    raise EOFError()
2023-06-24T14:50:32.613 ERROR:paramiko.transport:EOFError
2023-06-24T14:50:32.613 ERROR:paramiko.transport:
2023-06-24T14:50:32.614 ERROR:paramiko.transport:During handling of the above exception, another exception occurred:
2023-06-24T14:50:32.614 ERROR:paramiko.transport:
2023-06-24T14:50:32.614 ERROR:paramiko.transport:Traceback (most recent call last):
2023-06-24T14:50:32.614 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2093, in run
2023-06-24T14:50:32.614 ERROR:paramiko.transport:    self._check_banner()
2023-06-24T14:50:32.615 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2274, in _check_banner
2023-06-24T14:50:32.615 ERROR:paramiko.transport:    raise SSHException(
2023-06-24T14:50:32.615 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
2023-06-24T14:50:32.615 ERROR:paramiko.transport:
2023-06-24T14:50:32.616 WARNING:teuthology.provision.fog.smithi154:Error reading SSH protocol banner
2023-06-24T14:50:32.679 ERROR:teuthology.orchestra.connection:Error authenticating with smithi161.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:50:38.685 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:50:38.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:51:03.186 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:51:08.194 WARNING:teuthology.provision.fog.smithi161:
2023-06-24T14:51:14.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:51:18.260 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:51:20.304 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:51:23.250 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:51:25.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:51:30.432 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:51:35.438 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-24T14:51:38.512 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:51:41.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:51:44.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:51:50.512 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:51:52.592 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161
2023-06-24T14:51:58.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:52:05.578 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:52:20.639 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:52:35.245 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-24T14:52:35.246 DEBUG:teuthology.orchestra.run.smithi161:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-24T14:52:40.705 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:52:45.713 WARNING:teuthology.provision.fog.smithi154:timed out
2023-06-24T14:52:51.716 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:52:54.564 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-24T14:52:54.564 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-24T14:52:54.869 INFO:teuthology.provision.fog.smithi154:Node is ready
2023-06-24T14:52:54.869 DEBUG:teuthology.orchestra.run.smithi154:> hostname
2023-06-24T14:52:54.889 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com
2023-06-24T14:52:54.890 DEBUG:teuthology.orchestra.run.smithi154:> grep smithi154.front.sepia.ceph.com /etc/hosts
2023-06-24T14:52:54.949 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154
2023-06-24T14:52:54.950 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154
2023-06-24T14:52:55.057 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hosts
2023-06-24T14:52:55.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/172.21.15.154/172.21.15.154/g' /etc/hosts
2023-06-24T14:52:55.160 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hostname
2023-06-24T14:52:55.231 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154
2023-06-24T14:52:55.300 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release
2023-06-24T14:52:55.359 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream"
2023-06-24T14:52:55.359 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8"
2023-06-24T14:52:55.360 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos"
2023-06-24T14:52:55.360 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora"
2023-06-24T14:52:55.360 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8"
2023-06-24T14:52:55.361 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8"
2023-06-24T14:52:55.361 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-24T14:52:55.361 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31"
2023-06-24T14:52:55.361 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-24T14:52:55.362 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/"
2023-06-24T14:52:55.362 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-24T14:52:55.362 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-24T14:52:55.362 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-24T14:52:55.363 INFO:teuthology.provision.fog.smithi154:Deploy complete!
2023-06-24T14:52:55.364 DEBUG:teuthology.parallel:result is None
2023-06-24T14:52:55.397 INFO:teuthology.provision.fog.smithi161:Node is ready
2023-06-24T14:52:55.398 DEBUG:teuthology.orchestra.run.smithi161:> hostname
2023-06-24T14:52:55.425 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com
2023-06-24T14:52:55.425 DEBUG:teuthology.orchestra.run.smithi161:> grep smithi161.front.sepia.ceph.com /etc/hosts
2023-06-24T14:52:55.484 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161
2023-06-24T14:52:55.485 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161
2023-06-24T14:52:55.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hosts
2023-06-24T14:52:55.629 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/172.21.15.161/172.21.15.161/g' /etc/hosts
2023-06-24T14:52:55.697 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hostname
2023-06-24T14:52:55.764 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161
2023-06-24T14:52:55.831 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release
2023-06-24T14:52:55.889 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream"
2023-06-24T14:52:55.890 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8"
2023-06-24T14:52:55.890 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos"
2023-06-24T14:52:55.890 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora"
2023-06-24T14:52:55.890 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8"
2023-06-24T14:52:55.890 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8"
2023-06-24T14:52:55.891 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-24T14:52:55.891 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31"
2023-06-24T14:52:55.891 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-24T14:52:55.891 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/"
2023-06-24T14:52:55.891 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-24T14:52:55.891 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-24T14:52:55.891 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-24T14:52:55.892 INFO:teuthology.provision.fog.smithi161:Deploy complete!
2023-06-24T14:52:55.892 DEBUG:teuthology.parallel:result is None
2023-06-24T14:53:00.777 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:53:19.337 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:53:39.415 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:53:59.514 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:54:19.598 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:54:39.684 DEBUG:teuthology.provision.fog.smithi084:FOG request: GET /task/active None
2023-06-24T14:54:44.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:54:52.848 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-06-24T14:54:58.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:55:06.928 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-06-24T14:55:12.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:55:17.552 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-06-24T14:55:23.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:55:26.608 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84
2023-06-24T14:55:32.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:56:37.614 WARNING:teuthology.provision.fog.smithi084:timed out
2023-06-24T14:56:43.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:56:48.814 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-24T14:56:48.815 DEBUG:teuthology.orchestra.run.smithi084:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-24T14:56:49.100 INFO:teuthology.provision.fog.smithi084:Node is ready
2023-06-24T14:56:49.101 DEBUG:teuthology.orchestra.run.smithi084:> hostname
2023-06-24T14:56:49.120 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com
2023-06-24T14:56:49.120 DEBUG:teuthology.orchestra.run.smithi084:> grep smithi084.front.sepia.ceph.com /etc/hosts
2023-06-24T14:56:49.180 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084
2023-06-24T14:56:49.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname smithi084
2023-06-24T14:56:49.296 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/smithi084.front.sepia.ceph.com/smithi084/g' /etc/hosts
2023-06-24T14:56:49.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/172.21.15.84/172.21.15.84/g' /etc/hosts
2023-06-24T14:56:49.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/smithi084.front.sepia.ceph.com/smithi084/g' /etc/hostname
2023-06-24T14:56:49.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname smithi084
2023-06-24T14:56:49.535 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release
2023-06-24T14:56:49.594 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream"
2023-06-24T14:56:49.594 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8"
2023-06-24T14:56:49.595 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos"
2023-06-24T14:56:49.595 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora"
2023-06-24T14:56:49.595 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8"
2023-06-24T14:56:49.596 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8"
2023-06-24T14:56:49.596 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-24T14:56:49.596 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31"
2023-06-24T14:56:49.596 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-24T14:56:49.596 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/"
2023-06-24T14:56:49.596 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-24T14:56:49.596 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-24T14:56:49.597 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-24T14:56:49.597 INFO:teuthology.provision.fog.smithi084:Deploy complete!
2023-06-24T14:56:49.598 DEBUG:teuthology.parallel:result is None
2023-06-24T14:57:25.751 ERROR:teuthology.misc:write (smithi154.front.sepia.ceph.com): No route to host
2023-06-24T14:57:43.999 ERROR:teuthology.misc:write (smithi154.front.sepia.ceph.com): No route to host
2023-06-24T14:57:44.000 ERROR:teuthology.misc:write (smithi154.front.sepia.ceph.com): No route to host
2023-06-24T14:57:45.969 ERROR:teuthology.misc:write (smithi154.front.sepia.ceph.com): No route to host
2023-06-24T14:57:45.970 ERROR:teuthology.misc:write (smithi154.front.sepia.ceph.com): No route to host
2023-06-24T14:57:45.970 ERROR:teuthology.misc:write (smithi154.front.sepia.ceph.com): No route to host
2023-06-24T14:58:27.109 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com
2023-06-24T14:58:27.110 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-24T14:58:32.153 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com
2023-06-24T14:58:32.153 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-24T14:58:34.519 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com
2023-06-24T14:58:34.520 INFO:teuthology.lock.ops:New key found. Updating...
2023-06-24T14:58:34.537 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server
2023-06-24T14:58:34.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com'
2023-06-24T14:58:34.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:58:34.609 DEBUG:teuthology.orchestra.run.smithi084:> true
2023-06-24T14:58:34.744 DEBUG:teuthology.orchestra.run.smithi084:> uname -m
2023-06-24T14:58:34.802 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64
2023-06-24T14:58:34.803 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release
2023-06-24T14:58:34.864 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream"
2023-06-24T14:58:34.864 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8"
2023-06-24T14:58:34.865 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos"
2023-06-24T14:58:34.865 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora"
2023-06-24T14:58:34.865 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8"
2023-06-24T14:58:34.866 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8"
2023-06-24T14:58:34.866 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 8"
2023-06-24T14:58:34.867 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31"
2023-06-24T14:58:34.867 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:8"
2023-06-24T14:58:34.867 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/"
2023-06-24T14:58:34.868 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2023-06-24T14:58:34.868 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
2023-06-24T14:58:34.868 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2023-06-24T14:58:34.869 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server
2023-06-24T14:58:34.894 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server
2023-06-24T14:58:34.895 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com'
2023-06-24T14:58:34.895 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-24T14:58:40.017 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:58:44.062 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:58:49.806 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-24T14:58:50.809 ERROR:paramiko.transport:Exception (client): Error reading SSH protocol banner
2023-06-24T14:58:50.809 ERROR:paramiko.transport:Traceback (most recent call last):
2023-06-24T14:58:50.810 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2270, in _check_banner
2023-06-24T14:58:50.810 ERROR:paramiko.transport:    buf = self.packetizer.readline(timeout)
2023-06-24T14:58:50.810 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 380, in readline
2023-06-24T14:58:50.811 ERROR:paramiko.transport:    buf += self._read_timeout(timeout)
2023-06-24T14:58:50.811 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 609, in _read_timeout
2023-06-24T14:58:50.811 ERROR:paramiko.transport:    raise EOFError()
2023-06-24T14:58:50.811 ERROR:paramiko.transport:EOFError
2023-06-24T14:58:50.812 ERROR:paramiko.transport:
2023-06-24T14:58:50.812 ERROR:paramiko.transport:During handling of the above exception, another exception occurred:
2023-06-24T14:58:50.812 ERROR:paramiko.transport:
2023-06-24T14:58:50.813 ERROR:paramiko.transport:Traceback (most recent call last):
2023-06-24T14:58:50.813 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2093, in run
2023-06-24T14:58:50.813 ERROR:paramiko.transport:    self._check_banner()
2023-06-24T14:58:50.814 ERROR:paramiko.transport:  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2274, in _check_banner
2023-06-24T14:58:50.814 ERROR:paramiko.transport:    raise SSHException(
2023-06-24T14:58:50.814 ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
2023-06-24T14:58:50.815 ERROR:paramiko.transport:
2023-06-24T14:58:50.815 DEBUG:teuthology.orchestra.remote:Error reading SSH protocol banner
2023-06-24T14:58:50.816 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 327, in reimage_machines
    update_nodes(reimaged)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 38, in update_nodes
    inventory_info = remote.inventory_info
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 679, in inventory_info
    node['arch'] = self.arch
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 335, in arch
    self._arch = self.sh('uname -m').strip()
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 96, in sh
    proc = self.run(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 524, in run
    raise Exception(f'Cannot connect to remote host {self.shortname}')
Exception: Cannot connect to remote host smithi154
2023-06-24T14:58:50.817 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-24T14:58:50.817 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-24T14:58:50.899 INFO:teuthology.nuke:Checking targets against current locks
2023-06-24T14:58:50.964 DEBUG:teuthology.nuke:shortname: smithi084
2023-06-24T14:58:50.964 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-24T14:58:51.000 DEBUG:teuthology.nuke:shortname: smithi154
2023-06-24T14:58:51.001 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-24T14:58:51.005 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-23_21:02:01-rbd-reef-distro-default-smithi/7315323', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-24 14:46:25.059843', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPaOeWK+7VIzZaY474f8qjXIdNOtCsFOwoOxMxr1gHz7DrAXfU7YMcsNEyNUe1kT1K1VJy8cLfyPgcvYT71hLOU='}
2023-06-24T14:58:51.033 DEBUG:teuthology.nuke:shortname: smithi161
2023-06-24T14:58:51.034 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-24T14:58:51.037 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-23_21:02:01-rbd-reef-distro-default-smithi/7315337', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-24 14:54:16.187122', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyvu3i421Qbp2UPNz7VM4geMz/dBByMaWccC9KgF2YgxYPkjXxeu4Gll91yo852djp0IGw8FkikAcT/33v0iE4mxvpfZFCcq04NC1JcwLI4XvpekmhH5TMMTsKEqOaK212wJtTH3HCW3KT8fu1YvQdw6mQa67WyCfBVolBWE0t7xUUj+oNyj3EL32nMYS10kXhlTKbif1kZqdGvqzvrKYGCU+vmTga81Fa9M3t1zbVFPtpaMZB/Zf7YMi7pvrrkQF//1wwmn5z/+qB2kAQJtVf7r61HwOfvXmRy6AQbzh0VeHOsR8FmC8uHtIQIQx1u9w/Qo1trc/ebwWZ+dT5T6beUmZidDKYLU9XnKRMgzVKxMXSwDzvfT8U7jZwtumy4JxIaeeeErJph30KZiWBRpDXZzM2GwJ73HfubbOKZZx1BXvJ30HGb2cdGoggAxtgm2O2pvDarHGlRYTDbhOYd/Qj3bqfti2cm+0LioK9PF0SXPNKwTzJNx3i4RQ+MPownT0='}
2023-06-24T14:58:51.067 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-23_21:02:01-rbd-reef-distro-default-smithi/7315323', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-24 14:46:25.061098', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdp9d81BIs3+et+Z0537PlPdSjZwGf5pM0YbUG65iSaANZ4hdm2tQQqDlyhHaZN2caDa/Wa2td+bIW9DWKZaYYfhRJw423yc784/IQ02CvBZZvX/iFMmc/Wmt18W9KQikQa5wOpwBuvtc0rfrLRJYIE1rohoTy7MCF14mPBzuYbTm/L2xNSwk8Tpf2PQyeTIqVY/FvVYNM417ARBQmj5Yy94LFfroIsRqU5+PDmSlWhkV/NBEwSm5Fzv26BEbnsg3RM3EwcDJvtJglTGhQgwIHw0x1pQN7bOXs7vVkUF/Qv4fPNur3gWEl9C4aFk7AYBkqrAyHZDsaKGLOw275GUPrAts+VHcR9ezHxjCDqubCkD8G+5QhPZwbJR61GGFOS284QkXZMoZ9FcvfHqMIqXlGTeyd+sRRM82WJMjToGEslSv6S80nWjrZNlFNGOFdMUAK96SgMdkjKP9uWGKYbjW7aV7w8KtnyA4of13fKd1J9jVeoUwR+gKCRJZIzzz2jm8='}
2023-06-24T14:58:51.077 INFO:teuthology.orchestra.console:Power off smithi084
2023-06-24T14:58:51.077 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-24T14:58:51.105 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:58:51.124 INFO:teuthology.orchestra.console:Power off smithi154
2023-06-24T14:58:51.125 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-24T14:58:51.148 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:58:51.170 INFO:teuthology.orchestra.console:Power off smithi161
2023-06-24T14:58:51.171 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-24T14:58:51.193 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:58:55.141 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:58:55.182 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:58:55.232 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:58:59.253 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:00.296 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:00.344 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:04.366 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:04.580 INFO:teuthology.orchestra.console:Power off for smithi084 completed
2023-06-24T14:59:08.998 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:09.010 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:09.072 INFO:teuthology.lock.ops:unlocked: smithi084.front.sepia.ceph.com
2023-06-24T14:59:09.219 INFO:teuthology.orchestra.console:Power off for smithi161 completed
2023-06-24T14:59:09.382 INFO:teuthology.lock.ops:unlocked: smithi161.front.sepia.ceph.com
2023-06-24T14:59:14.112 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:19.226 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:24.338 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-24T14:59:24.551 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2023-06-24T14:59:26.099 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com
2023-06-24T14:59:26.467 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 57, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 327, in reimage_machines
    update_nodes(reimaged)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/lock/ops.py", line 38, in update_nodes
    inventory_info = remote.inventory_info
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 679, in inventory_info
    node['arch'] = self.arch
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 335, in arch
    self._arch = self.sh('uname -m').strip()
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 96, in sh
    proc = self.run(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/orchestra/remote.py", line 524, in run
    raise Exception(f'Cannot connect to remote host {self.shortname}')
Exception: Cannot connect to remote host smithi154