2023-06-28T08:49:10.499 INFO:root:teuthology version: 0.0.1.dev132+gaa9345a
2023-06-28T08:49:10.501 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-28T08:49:10.501 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-28T08:49:35.710 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7ffaebfbb670>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7ffaeb9633a0>, signals=[15])
2023-06-28T08:49:35.713 INFO:teuthology.lock.ops:Start node 'smithi061.front.sepia.ceph.com' reimaging
2023-06-28T08:49:35.713 INFO:teuthology.lock.ops:Updating [smithi061.front.sepia.ceph.com]: reset os type and version on server
2023-06-28T08:49:35.714 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server
2023-06-28T08:49:39.038 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging
2023-06-28T08:49:39.039 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server
2023-06-28T08:49:39.039 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2023-06-28T08:49:39.040 DEBUG:teuthology.provision.fog.smithi061:FOG request: GET /host {"name": "smithi061"}
2023-06-28T08:49:44.063 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /host {"name": "smithi154"}
2023-06-28T08:49:49.917 DEBUG:teuthology.provision.fog.smithi061:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-06-28T08:49:59.953 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /image {"name": "smithi_rhel_8.6"}
2023-06-28T08:50:04.955 DEBUG:teuthology.provision.fog.smithi061:Requesting image smithi_rhel_8.6 (ID 5)
2023-06-28T08:50:04.956 DEBUG:teuthology.provision.fog.smithi061:FOG request: PUT /host/166 {"imageID": 5}
2023-06-28T08:50:10.002 DEBUG:teuthology.provision.fog.smithi154:Requesting image smithi_rhel_8.6 (ID 5)
2023-06-28T08:50:10.003 DEBUG:teuthology.provision.fog.smithi154:FOG request: PUT /host/259 {"imageID": 5}
2023-06-28T08:50:15.010 INFO:teuthology.provision.fog.smithi061:Scheduling deploy of rhel 8.6
2023-06-28T08:50:15.010 DEBUG:teuthology.provision.fog.smithi061:FOG request: GET /task/active None
2023-06-28T08:50:15.074 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of rhel 8.6
2023-06-28T08:50:15.074 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-28T08:50:15.134 DEBUG:teuthology.provision.fog.smithi061:Canceling deploy task with ID 350446
2023-06-28T08:50:15.135 DEBUG:teuthology.provision.fog.smithi061:FOG request: DELETE /task/cancel {"id": 350446}
2023-06-28T08:50:15.165 DEBUG:teuthology.provision.fog.smithi154:Canceling deploy task with ID 350447
2023-06-28T08:50:15.166 DEBUG:teuthology.provision.fog.smithi154:FOG request: DELETE /task/cancel {"id": 350447}
2023-06-28T08:50:20.174 DEBUG:teuthology.provision.fog.smithi061:FOG request: GET /tasktype {"name": "deploy"}
2023-06-28T08:50:25.227 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /tasktype {"name": "deploy"}
2023-06-28T08:50:30.230 DEBUG:teuthology.provision.fog.smithi061:FOG request: POST /host/166/task {"taskTypeID": 1}
2023-06-28T08:50:30.281 DEBUG:teuthology.provision.fog.smithi154:FOG request: POST /host/259/task {"taskTypeID": 1}
2023-06-28T08:50:30.354 DEBUG:teuthology.provision.fog.smithi061:FOG request: GET /task/active None
2023-06-28T08:50:35.362 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-28T08:50:40.492 INFO:teuthology.orchestra.console:Power off smithi061
2023-06-28T08:50:40.494 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-28T08:50:40.513 INFO:teuthology.orchestra.console:Power off smithi154
2023-06-28T08:50:40.514 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-28T08:50:40.525 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:50:40.538 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:50:40.655 INFO:teuthology.orchestra.console:Power off for smithi061 completed
2023-06-28T08:50:40.670 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2023-06-28T08:50:40.757 INFO:teuthology.orchestra.console:Power on smithi061
2023-06-28T08:50:40.757 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-28T08:50:40.771 INFO:teuthology.orchestra.console:Power on smithi154
2023-06-28T08:50:40.772 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-28T08:50:40.784 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:50:40.797 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:50:44.812 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:50:44.822 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:50:48.928 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:50:48.936 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:50:49.139 INFO:teuthology.orchestra.console:Power on for smithi061 completed
2023-06-28T08:50:49.146 INFO:teuthology.orchestra.console:Power on for smithi154 completed
2023-06-28T08:50:49.240 INFO:teuthology.provision.fog.smithi061:Waiting for deploy to finish
2023-06-28T08:50:49.241 DEBUG:teuthology.provision.fog.smithi061:FOG request: GET /task/active None
2023-06-28T08:50:54.249 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish
2023-06-28T08:50:54.250 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None
2023-06-28T08:50:59.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:50:59.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:51:02.448 WARNING:teuthology.provision.fog.smithi061:[Errno None] Unable to connect to port 22 on 172.21.15.61
2023-06-28T08:51:08.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:51:16.529 WARNING:teuthology.provision.fog.smithi061:[Errno None] Unable to connect to port 22 on 172.21.15.61
2023-06-28T08:51:17.748 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-28T08:51:22.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:51:27.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:51:32.541 WARNING:teuthology.provision.fog.smithi061:[Errno None] Unable to connect to port 22 on 172.21.15.61
2023-06-28T08:51:35.604 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-28T08:51:38.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:51:41.608 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:51:46.615 WARNING:teuthology.provision.fog.smithi061:[Errno None] Unable to connect to port 22 on 172.21.15.61
2023-06-28T08:51:49.680 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-28T08:51:52.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:51:57.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:52:02.629 WARNING:teuthology.provision.fog.smithi061:[Errno None] Unable to connect to port 22 on 172.21.15.61
2023-06-28T08:52:08.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:52:14.736 WARNING:teuthology.provision.fog.smithi061:[Errno None] Unable to connect to port 22 on 172.21.15.61
2023-06-28T08:52:20.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:52:25.744 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2023-06-28T08:52:28.816 WARNING:teuthology.provision.fog.smithi061:[Errno None] Unable to connect to port 22 on 172.21.15.61
2023-06-28T08:52:31.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:52:36.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T08:52:41.839 ERROR:teuthology.orchestra.connection:Error authenticating with smithi061.front.sepia.ceph.com: Authentication failed.
2023-06-28T08:52:41.841 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-28T08:52:41.842 DEBUG:teuthology.orchestra.run.smithi061:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-28T08:52:41.842 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:41.873 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed.
2023-06-28T08:52:41.874 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-28T08:52:41.875 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-28T08:52:41.876 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:42.853 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:42.887 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:43.860 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:43.897 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:44.868 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:44.906 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:45.878 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:45.916 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:46.886 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:46.925 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:47.896 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:47.935 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:48.905 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:48.944 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:49.915 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:49.954 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:50.924 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:50.963 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:51.933 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:51.973 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:52.944 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:52.983 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:53.955 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:53.994 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:54.964 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:55.003 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:55.975 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:56.013 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:56.986 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:57.023 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:57.997 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:58.032 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:59.007 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:52:59.041 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:00.016 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:00.051 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:01.026 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:01.059 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:02.036 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:02.069 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:03.047 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:03.080 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:04.059 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:04.089 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:05.068 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:05.099 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:06.077 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:06.110 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:07.087 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:07.122 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:08.097 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:08.131 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:09.108 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:09.140 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:10.117 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:10.149 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:11.129 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:11.159 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:12.138 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:12.169 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:13.148 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:13.178 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:14.158 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:14.187 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:15.168 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:15.196 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:16.176 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:16.205 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:17.184 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:17.215 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:18.193 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:18.224 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:19.202 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:19.232 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:20.212 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:20.242 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:21.222 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:21.251 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:22.232 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:22.261 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:23.241 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:23.270 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:24.250 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:24.280 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:25.260 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:25.289 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:26.271 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:26.299 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:27.280 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:27.309 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:28.290 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:28.319 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:29.299 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:29.328 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:30.312 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:30.338 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:31.348 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:53:32.360 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T08:55:45.520 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-06-28T08:55:45.524 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 295, in _wait_for_ready
    self.remote.run(args=cmd, timeout=600)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/remote.py", line 525, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 450, in run
    r.execute()
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 97, in execute
    self.client.exec_command(self.command, timeout=self.timeout)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 510, in exec_command
    chan = self._transport.open_session(timeout=timeout)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 919, in open_session
    return self.open_channel(
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1050, in open_channel
    raise e
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run
    ptype, m = self.packetizer.read_message()
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message
    header = self.read_all(self.__block_size_in, check_rekey=True)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all
    x = self.__socket.recv(n)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv
    return self._sock.recv(*args)
ConnectionResetError: [Errno 104] Connection reset by peer
2023-06-28T08:55:45.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-28T08:55:45.525 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-28T08:55:47.327 INFO:teuthology.nuke:Checking targets against current locks
2023-06-28T08:55:47.387 DEBUG:teuthology.nuke:shortname: smithi061
2023-06-28T08:55:47.388 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-28T08:55:47.411 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi061.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_22:53:08-rados-reef-release-distro-default-smithi/7319334', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 08:48:54.694414', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDeN6aSV+qQ6v67p8Fd1xYT3P0JV+lLqUFNvVMfJoGGlxfRAd3WfOZiFlRX1k5UCT3qFELHsw2YBwVZuPLDXK6iQRAT+uoXsQupsasBR09bVAysod/s3SsiX6J/4i5IhL31PB7tpeZ8EpVI5muyalpJ6d4FLlLsNuwWyC/b+akoXG25efYiBkaCToOrknOqszxhWUtCCHX0+6b1Zx+UVBIKdp3FSyI+DhJ9a2+5/zyWIAyIfj4nvR7M+6HEPaxJNCgRavQHKuFRDu54AKQY+dJRHX1AAGfFQysHWlLJJL2jjXtAF2POWP8lnv7Vo1RN7sbL15la9O/kE2cfl2St4f+GmFCupn7i4nGpjvnL87Ak3ge13120th6sn+nkApucteOm0/b/mcXBXxcfeJM+xrbVq0Y/445LxSYCsjz/lnhfFlcc7BeMlKVpPew5HGXmS9t8Dok2ifgTYRG8BN4DR+kjwOaYFb8DSjhSQGa6B+6KLEHingPvNuTfijlbMpfcBoE='}
2023-06-28T08:55:47.416 DEBUG:teuthology.nuke:shortname: smithi154
2023-06-28T08:55:47.417 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-28T08:55:47.446 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_22:53:08-rados-reef-release-distro-default-smithi/7319334', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 08:48:54.692535', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdMvcSDOth4L4i60pNQOayn6tXHafo6isphnhsgDyuYQ0XRBbxG1nd/JxMzxqrtAt780b7WgJOBqpTyzuTAQ+1oE2i38k8VuKtgx3PedLJTGVKjZwoxHrFEMqw7N9f0kZJEkRaOD+RCct6vijibscfNGOqPfsaJhWahtFGwpAO10tYawjFRr7rIlcQEf3G9r00Nq/kAT1y0HPpvtfRNTRTR7pI9CF6I9D3A4IswQgYm4Bj1tJeWEMfyV1t1kikM/X0LKo/fPeijvvBZDuLuPbTXAYAJkmubSOzkGdNpwD087RcIcKN4CYfotTWc9e+yDREcBuVxX/YbHqYUbGR73vKEqTtI4IGPh7lR57DmQ/H+YTmFo6ulAmLCgMr4iVLKqVN99pnVU4vqHklyLDXE4z2qg/F+fiBCaYhGISnBOqnGKZ+sWcjVsab/6gYyMTkHpB+toHpkVTr5MwMxZpOQcQoY+5dY4YjhG0y1KA4Ku6xATr6zDrZfBEH90nhG15Ruic='}
2023-06-28T08:55:47.472 INFO:teuthology.orchestra.console:Power off smithi061
2023-06-28T08:55:47.473 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-28T08:55:47.495 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:55:47.517 INFO:teuthology.orchestra.console:Power off smithi154
2023-06-28T08:55:47.517 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-28T08:55:47.542 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:55:47.568 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-06-28T08:55:51.531 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:55:51.574 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:55:55.648 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:55:55.686 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:00.761 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:00.798 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:05.876 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:05.910 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:10.989 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:11.023 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:11.200 INFO:teuthology.orchestra.console:Power off for smithi061 completed
2023-06-28T08:56:15.462 INFO:teuthology.lock.ops:unlocked: smithi061.front.sepia.ceph.com
2023-06-28T08:56:19.416 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:24.528 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:29.640 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:34.753 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:39.865 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T08:56:40.078 INFO:teuthology.orchestra.console:Power off for smithi154 completed
2023-06-28T08:56:50.248 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com
2023-06-28T08:57:00.510 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 57, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 295, in _wait_for_ready
    self.remote.run(args=cmd, timeout=600)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/remote.py", line 525, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 450, in run
    r.execute()
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 97, in execute
    self.client.exec_command(self.command, timeout=self.timeout)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 510, in exec_command
    chan = self._transport.open_session(timeout=timeout)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 919, in open_session
    return self.open_channel(
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1050, in open_channel
    raise e
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run
    ptype, m = self.packetizer.read_message()
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message
    header = self.read_all(self.__block_size_in, check_rekey=True)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all
    x = self.__socket.recv(n)
  File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv
    return self._sock.recv(*args)
ConnectionResetError: [Errno 104] Connection reset by peer