2023-06-28T02:59:53.837 INFO:root:teuthology version: 0.0.1.dev130+g3fdf1f2
2023-06-28T02:59:53.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-28T02:59:53.840 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-28T02:59:53.997 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7fab37118700>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7fab383358b0>, signals=[15])
2023-06-28T02:59:53.998 INFO:teuthology.lock.ops:Start node 'smithi111.front.sepia.ceph.com' reimaging
2023-06-28T02:59:53.999 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: reset os type and version on server
2023-06-28T02:59:53.999 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server
2023-06-28T02:59:54.019 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging
2023-06-28T02:59:54.019 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server
2023-06-28T02:59:54.019 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server
2023-06-28T02:59:54.021 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /host {"name": "smithi111"}
2023-06-28T03:00:04.059 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /host {"name": "smithi149"}
2023-06-28T03:00:09.094 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-28T03:00:12.776 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /image {"name": "smithi_centos_8.stream"}
2023-06-28T03:00:12.806 DEBUG:teuthology.provision.fog.smithi111:Requesting image smithi_centos_8.stream (ID 3)
2023-06-28T03:00:12.807 DEBUG:teuthology.provision.fog.smithi111:FOG request: PUT /host/216 {"imageID": 3}
2023-06-28T03:00:12.819 DEBUG:teuthology.provision.fog.smithi149:Requesting image smithi_centos_8.stream (ID 3)
2023-06-28T03:00:12.820 DEBUG:teuthology.provision.fog.smithi149:FOG request: PUT /host/254 {"imageID": 3}
2023-06-28T03:00:12.850 INFO:teuthology.provision.fog.smithi111:Scheduling deploy of centos 8.stream
2023-06-28T03:00:12.850 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /task/active None
2023-06-28T03:00:12.865 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of centos 8.stream
2023-06-28T03:00:12.865 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-28T03:00:12.897 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /tasktype {"name": "deploy"}
2023-06-28T03:00:12.907 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /tasktype {"name": "deploy"}
2023-06-28T03:00:12.934 DEBUG:teuthology.provision.fog.smithi111:FOG request: POST /host/216/task {"taskTypeID": 1}
2023-06-28T03:00:12.943 DEBUG:teuthology.provision.fog.smithi149:FOG request: POST /host/254/task {"taskTypeID": 1}
2023-06-28T03:00:12.992 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /task/active None
2023-06-28T03:00:13.008 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-28T03:00:24.279 INFO:teuthology.orchestra.console:Power off smithi111
2023-06-28T03:00:24.280 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-28T03:00:24.304 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:24.348 INFO:teuthology.orchestra.console:Power off smithi149
2023-06-28T03:00:24.349 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-28T03:00:24.372 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:28.328 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:28.396 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:32.440 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:32.508 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:37.550 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:37.619 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:42.663 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:42.729 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:42.873 INFO:teuthology.orchestra.console:Power off for smithi111 completed
2023-06-28T03:00:42.974 INFO:teuthology.orchestra.console:Power on smithi111
2023-06-28T03:00:42.974 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-28T03:00:42.995 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:47.021 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:47.232 INFO:teuthology.orchestra.console:Power on for smithi111 completed
2023-06-28T03:00:47.333 INFO:teuthology.provision.fog.smithi111:Waiting for deploy to finish
2023-06-28T03:00:47.333 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /task/active None
2023-06-28T03:00:52.341 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:00:52.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:00:57.511 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2023-06-28T03:00:57.612 INFO:teuthology.orchestra.console:Power on smithi149
2023-06-28T03:00:57.613 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2023-06-28T03:01:02.610 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:01:07.624 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:01:12.735 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:01:12.950 INFO:teuthology.orchestra.console:Power on for smithi149 completed
2023-06-28T03:01:13.051 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish
2023-06-28T03:01:13.052 DEBUG:teuthology.provision.fog.smithi149:FOG request: GET /task/active None
2023-06-28T03:01:13.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:01:15.760 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2023-06-28T03:01:21.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:01:29.844 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2023-06-28T03:01:31.632 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-28T03:01:35.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:01:37.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:01:42.642 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2023-06-28T03:01:45.712 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-28T03:01:48.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:01:51.700 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2023-06-28T03:01:51.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:01:54.768 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-28T03:01:57.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:02:02.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:02:07.708 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2023-06-28T03:02:10.772 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-28T03:02:13.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:02:16.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:02:19.856 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-28T03:02:25.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:02:32.048 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111
2023-06-28T03:02:38.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:02:38.165 ERROR:teuthology.orchestra.connection:Error authenticating with smithi111.front.sepia.ceph.com: Authentication failed.
2023-06-28T03:02:38.167 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-28T03:02:38.168 DEBUG:teuthology.orchestra.run.smithi111:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-28T03:02:38.168 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:39.173 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:40.178 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:41.183 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:42.188 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:43.194 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:44.199 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:45.205 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:46.210 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:47.216 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:48.221 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:49.200 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149
2023-06-28T03:02:49.226 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:50.232 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:51.237 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:52.242 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:53.247 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:54.252 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:02:55.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2023-06-28T03:03:00.205 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:00.278 ERROR:teuthology.orchestra.connection:Error authenticating with smithi149.front.sepia.ceph.com: Authentication failed.
2023-06-28T03:03:00.279 INFO:teuthology.orchestra.run:Running command with timeout 600
2023-06-28T03:03:00.279 DEBUG:teuthology.orchestra.run.smithi149:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2023-06-28T03:03:00.280 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:01.213 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:01.287 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:02.221 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:02.297 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:03.230 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:03.315 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:04.238 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:04.324 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:05.248 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:05.332 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:06.256 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:06.340 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:07.265 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:07.349 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:08.273 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:08.358 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:09.282 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:09.366 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:10.291 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:10.375 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:11.299 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:11.384 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:12.308 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:12.392 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:13.316 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:13.401 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:14.324 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:14.408 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:15.332 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:15.416 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:16.340 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:16.424 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:17.347 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:17.431 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:18.355 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:18.439 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:19.363 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:19.447 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:20.371 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:20.455 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:21.380 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:21.464 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:22.388 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:22.472 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:23.397 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:23.481 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:24.406 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:24.489 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:25.414 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:25.497 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:26.423 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:26.507 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:27.432 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:27.515 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:28.440 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:28.523 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:29.448 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:29.531 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:30.456 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:30.539 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:31.465 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:31.548 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:32.474 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:32.557 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:33.482 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:33.566 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:34.491 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:34.574 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:35.584 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:36.593 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:37.602 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:38.611 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:39.620 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:40.629 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:41.638 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:42.646 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:43.656 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:03:44.666 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented')
2023-06-28T03:05:33.044 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104)
2023-06-28T03:05:33.048 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines...
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/provision/fog.py", line 295, in _wait_for_ready
    self.remote.run(args=cmd, timeout=600)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/remote.py", line 525, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 450, in run
    r.execute()
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 97, in execute
    self.client.exec_command(self.command, timeout=self.timeout)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 510, in exec_command
    chan = self._transport.open_session(timeout=timeout)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 919, in open_session
    return self.open_channel(
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1050, in open_channel
    raise e
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run
    ptype, m = self.packetizer.read_message()
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message
    header = self.read_all(self.__block_size_in, check_rekey=True)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all
    x = self.__socket.recv(n)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv
    return self._sock.recv(*args)
ConnectionResetError: [Errno 104] Connection reset by peer
2023-06-28T03:05:33.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2023-06-28T03:05:33.050 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None)
2023-06-28T03:05:38.151 INFO:teuthology.nuke:Checking targets against current locks
2023-06-28T03:05:43.228 DEBUG:teuthology.nuke:shortname: smithi111
2023-06-28T03:05:43.229 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-28T03:05:43.245 DEBUG:teuthology.nuke:shortname: smithi149
2023-06-28T03:05:43.246 INFO:teuthology.task.internal.check_lock:Checking locks...
2023-06-28T03:05:43.250 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_20:20:53-fs-quincy_release-distro-default-smithi/7318700', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 02:59:53.024191', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDn/hRNYhRAL6AO1aqMQfCPX67W3L+MBN97RYqFx2YI/IlpJXrFOgkU5L1JmFquUHf3n1MKxikM75B+2tm1BJGmdmFo6RP62iZErso0WGpgIkWhWD5wHEs+bN2gf+D/7dZcTVbtmIUCU1ThCbSwIThl7DNSWcgyp7Z0U6S5KtUtSOZdIiPU5eIEQAAN/C3mDY4VC/5jroaNap9/6HAWL7Lz7HzTisFkvmBn9MqiRn5wTTOblpM4pYkrMHGGoyc5RuuL/aQ0+h3yiPbLxz3xgU/wRTNz6ZeqxcKeFXqCNSixli9QhOlGKkg0QQUvkN3fknrnN0Z5zsa0gtHjdO/NHDz+4u5a7hksMW6D3uWXlmtPT64Gxtu0Wo6/l58gy22zVprFhvj1a4uaiPoyAw1asuW3iJm3CQSGuOvB6ERFPOAWyvrzpM5mK/sMPFCUKj36sbbMoE7CU4xhwMnN1hlTh1WZyLBELjZQki8Oinuv+iPCvisDusVDqr6RRw2YT75bg6s='}
2023-06-28T03:05:43.266 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_20:20:53-fs-quincy_release-distro-default-smithi/7318700', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 02:59:53.021227', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHHyt62kZ/aSuCvtnNZWty05XiM6i1ZhcLitdo0MB8E8TjfYUREgoej4YxefeUIDzc82NPZ6nQUAZg2IUvcl+UY='}
2023-06-28T03:05:43.294 INFO:teuthology.orchestra.console:Power off smithi111
2023-06-28T03:05:43.295 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-28T03:05:43.310 INFO:teuthology.orchestra.console:Power off smithi149
2023-06-28T03:05:43.310 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2023-06-28T03:05:43.321 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:05:43.334 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:05:47.349 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:05:47.359 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:05:51.462 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:05:51.470 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:05:51.682 INFO:teuthology.orchestra.console:Power off for smithi149 completed
2023-06-28T03:05:51.836 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com
2023-06-28T03:05:55.573 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2023-06-28T03:05:55.783 INFO:teuthology.orchestra.console:Power off for smithi111 completed
2023-06-28T03:05:55.944 INFO:teuthology.lock.ops:unlocked: smithi111.front.sepia.ceph.com
2023-06-28T03:05:56.167 CRITICAL:teuthology:Uncaught exception
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/bin/teuthology-dispatcher", line 8, in <module>
    sys.exit(main())
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/scripts/dispatcher.py", line 35, in main
    sys.exit(teuthology.dispatcher.main(args))
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/dispatcher/__init__.py", line 69, in main
    return supervisor.main(args)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/dispatcher/supervisor.py", line 57, in main
    reimage(job_config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/dispatcher/supervisor.py", line 227, in reimage
    reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/lock/ops.py", line 325, in reimage_machines
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/provision/__init__.py", line 44, in reimage
    result = obj.create()
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/provision/fog.py", line 90, in create
    self._wait_for_ready()
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/provision/fog.py", line 295, in _wait_for_ready
    self.remote.run(args=cmd, timeout=600)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/remote.py", line 525, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 450, in run
    r.execute()
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/teuthology/orchestra/run.py", line 97, in execute
    self.client.exec_command(self.command, timeout=self.timeout)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 510, in exec_command
    chan = self._transport.open_session(timeout=timeout)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 919, in open_session
    return self.open_channel(
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1050, in open_channel
    raise e
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run
    ptype, m = self.packetizer.read_message()
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message
    header = self.read_all(self.__block_size_in, check_rekey=True)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all
    x = self.__socket.recv(n)
  File "/home/teuthworker/src/git.ceph.com_teuthology_3fdf1f2d90b4263fafe8c0792229f37d37a2ab95/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv
    return self._sock.recv(*args)
ConnectionResetError: [Errno 104] Connection reset by peer