2023-06-28T04:53:25.338 INFO:root:teuthology version: 0.0.1.dev132+gaa9345a 2023-06-28T04:53:25.339 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T04:53:25.340 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T04:53:25.520 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f973edf63a0>, signals=[15]) 2023-06-28T04:53:25.522 INFO:teuthology.lock.ops:Start node 'smithi022.front.sepia.ceph.com' reimaging 2023-06-28T04:53:25.522 INFO:teuthology.lock.ops:Updating [smithi022.front.sepia.ceph.com]: reset os type and version on server 2023-06-28T04:53:25.523 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2023-06-28T04:53:25.543 INFO:teuthology.lock.ops:Start node 'smithi089.front.sepia.ceph.com' reimaging 2023-06-28T04:53:25.544 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: reset os type and version on server 2023-06-28T04:53:25.544 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2023-06-28T04:53:25.545 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /host {"name": "smithi022"} 2023-06-28T04:53:29.992 INFO:teuthology.lock.ops:Start node 'smithi111.front.sepia.ceph.com' reimaging 2023-06-28T04:53:29.993 INFO:teuthology.lock.ops:Updating [smithi111.front.sepia.ceph.com]: reset os type and version on server 2023-06-28T04:53:29.993 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2023-06-28T04:53:29.994 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /host {"name": "smithi089"} 2023-06-28T04:53:44.026 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /host {"name": "smithi111"} 2023-06-28T04:53:54.510 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-06-28T04:54:04.499 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-06-28T04:54:05.232 DEBUG:teuthology.provision.fog.smithi022:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-06-28T04:54:05.232 DEBUG:teuthology.provision.fog.smithi022:FOG request: PUT /host/128 {"imageID": 2} 2023-06-28T04:54:05.259 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /image {"name": "smithi_ubuntu_20.04"} 2023-06-28T04:54:05.277 INFO:teuthology.provision.fog.smithi022:Scheduling deploy of ubuntu 20.04 2023-06-28T04:54:05.277 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:54:05.283 DEBUG:teuthology.provision.fog.smithi089:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-06-28T04:54:05.283 DEBUG:teuthology.provision.fog.smithi089:FOG request: PUT /host/194 {"imageID": 2} 2023-06-28T04:54:05.294 DEBUG:teuthology.provision.fog.smithi111:Requesting image smithi_ubuntu_20.04 (ID 2) 2023-06-28T04:54:05.294 DEBUG:teuthology.provision.fog.smithi111:FOG request: PUT /host/216 {"imageID": 2} 2023-06-28T04:54:05.333 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /tasktype {"name": "deploy"} 2023-06-28T04:54:05.341 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of ubuntu 20.04 2023-06-28T04:54:05.341 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-06-28T04:54:05.344 INFO:teuthology.provision.fog.smithi111:Scheduling deploy of ubuntu 20.04 2023-06-28T04:54:05.344 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /task/active None 2023-06-28T04:54:05.364 DEBUG:teuthology.provision.fog.smithi022:FOG request: POST /host/128/task {"taskTypeID": 1} 2023-06-28T04:54:05.400 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /tasktype {"name": "deploy"} 2023-06-28T04:54:05.403 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /tasktype {"name": "deploy"} 2023-06-28T04:54:05.421 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:54:05.436 DEBUG:teuthology.provision.fog.smithi089:FOG request: POST /host/194/task {"taskTypeID": 1} 2023-06-28T04:54:05.438 DEBUG:teuthology.provision.fog.smithi111:FOG request: POST /host/216/task {"taskTypeID": 1} 2023-06-28T04:54:05.483 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /task/active None 2023-06-28T04:54:05.496 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-06-28T04:54:25.529 INFO:teuthology.orchestra.console:Power off smithi022 2023-06-28T04:54:25.531 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T04:54:25.554 INFO:teuthology.orchestra.console:Power off smithi111 2023-06-28T04:54:25.555 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T04:54:25.565 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:25.581 INFO:teuthology.orchestra.console:Power off smithi089 2023-06-28T04:54:25.582 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T04:54:25.592 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:25.603 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:25.692 INFO:teuthology.orchestra.console:Power off for smithi022 completed 2023-06-28T04:54:25.714 INFO:teuthology.orchestra.console:Power off for smithi111 completed 2023-06-28T04:54:25.794 INFO:teuthology.orchestra.console:Power on smithi022 2023-06-28T04:54:25.794 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-28T04:54:25.815 INFO:teuthology.orchestra.console:Power on smithi111 2023-06-28T04:54:25.815 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-28T04:54:25.822 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:25.833 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:29.630 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:29.842 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:29.853 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:30.061 INFO:teuthology.orchestra.console:Power on for smithi111 completed 2023-06-28T04:54:30.162 INFO:teuthology.provision.fog.smithi111:Waiting for deploy to finish 2023-06-28T04:54:30.163 DEBUG:teuthology.provision.fog.smithi111:FOG request: GET /task/active None 2023-06-28T04:54:30.231 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:54:33.296 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-06-28T04:54:33.954 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:34.164 INFO:teuthology.orchestra.console:Power on for smithi022 completed 2023-06-28T04:54:34.265 INFO:teuthology.provision.fog.smithi022:Waiting for deploy to finish 2023-06-28T04:54:34.266 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:54:35.997 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:39.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:54:44.308 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:47.376 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-06-28T04:54:49.419 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:54:51.072 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:54:56.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:55:01.081 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:55:04.144 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-06-28T04:55:06.192 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:55:10.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:55:15.153 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:55:16.170 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:55:21.178 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-06-28T04:55:21.280 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2023-06-28T04:55:21.381 INFO:teuthology.orchestra.console:Power on smithi089 2023-06-28T04:55:21.381 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-28T04:55:25.506 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:55:27.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:55:29.530 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:55:30.256 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-06-28T04:55:34.639 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:55:36.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:55:36.263 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:55:39.312 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-06-28T04:55:39.751 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:55:44.864 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T04:55:45.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:55:50.423 INFO:teuthology.orchestra.console:Power on for smithi089 completed 2023-06-28T04:55:50.524 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish 2023-06-28T04:55:50.524 DEBUG:teuthology.provision.fog.smithi089:FOG request: GET /task/active None 2023-06-28T04:55:50.604 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:55:51.350 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:55:56.357 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-06-28T04:56:02.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:56:09.072 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-06-28T04:56:10.452 WARNING:teuthology.provision.fog.smithi111:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-06-28T04:56:11.434 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:56:16.441 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:56:21.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:56:26.450 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-06-28T04:56:26.571 ERROR:teuthology.orchestra.connection:Error authenticating with smithi111.front.sepia.ceph.com: Authentication failed. 2023-06-28T04:56:26.574 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-28T04:56:26.575 DEBUG:teuthology.orchestra.run.smithi111:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-28T04:56:26.576 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:27.582 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:28.587 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:29.592 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:30.597 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:31.602 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:32.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:56:32.608 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:33.613 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:34.619 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:35.504 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-06-28T04:56:35.625 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:36.630 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:37.635 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:38.641 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:39.646 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:40.651 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:41.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:56:46.507 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:56:51.497 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-06-28T04:56:51.498 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:51.599 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:52.603 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:53.608 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:54.611 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:55.615 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:56.620 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:56:57.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:57:02.506 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:03.510 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:04.515 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:04.564 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2023-06-28T04:57:05.521 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:06.526 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:06.553 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:57:07.531 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:08.536 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:09.541 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:10.546 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:10.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:57:15.570 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:15.677 ERROR:teuthology.orchestra.connection:Error authenticating with smithi089.front.sepia.ceph.com: Authentication failed. 2023-06-28T04:57:15.677 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-28T04:57:15.678 DEBUG:teuthology.orchestra.run.smithi089:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-28T04:57:15.678 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:16.577 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:16.684 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:17.584 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:17.692 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:18.592 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:18.700 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:19.600 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:19.709 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:20.609 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:20.717 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:21.608 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:57:26.614 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:26.614 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:27.620 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:27.620 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:28.626 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:28.626 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:29.632 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:29.633 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:30.638 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:30.639 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:31.645 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:31.645 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:32.651 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:32.652 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:33.658 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:33.658 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:34.664 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:34.664 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:35.670 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:35.671 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:36.677 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:36.678 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:37.684 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:37.685 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:38.691 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:38.691 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:39.697 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:39.698 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:40.704 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:40.704 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:41.670 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:57:46.678 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:46.678 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:47.683 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:47.683 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:48.689 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:48.689 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:49.695 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:49.695 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:50.702 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:50.702 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:51.708 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:51.709 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:52.714 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:52.714 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:53.720 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:53.720 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:54.726 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:54.726 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:55.732 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:55.733 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:56.739 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:56.740 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:57.746 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:57.747 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:58.753 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:58.754 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:59.761 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:57:59.761 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:00.767 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:00.768 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:01.727 DEBUG:teuthology.provision.fog.smithi022:FOG request: GET /task/active None 2023-06-28T04:58:06.729 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:06.730 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:06.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:58:11.789 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:11.790 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:12.796 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:12.797 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:13.803 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:13.803 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:14.808 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:14.809 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:15.815 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:15.815 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:16.822 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:16.822 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:17.828 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:17.829 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:18.835 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:18.835 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:19.842 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:19.843 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:20.849 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:20.849 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:21.855 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:22.862 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:23.869 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:24.876 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:25.884 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:26.891 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:27.898 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:28.906 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:29.914 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:30.132 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-06-28T04:58:30.922 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:31.931 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:32.940 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:33.948 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:34.957 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:35.965 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:36.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:58:41.138 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:42.143 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:43.149 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:44.156 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:44.212 WARNING:teuthology.provision.fog.smithi022:[Errno None] Unable to connect to port 22 on 172.21.15.22 2023-06-28T04:58:45.162 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:46.168 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:47.174 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:48.180 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:49.187 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:50.193 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:50.214 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T04:58:51.200 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:52.207 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:53.214 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:54.220 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:55.227 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:56.233 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:57.240 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:58.246 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:58:59.253 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:00.259 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:01.265 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:02.272 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:03.279 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:04.285 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:05.292 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:06.298 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:07.305 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T04:59:50.216 WARNING:teuthology.provision.fog.smithi022:timed out 2023-06-28T04:59:56.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T05:00:01.424 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-28T05:00:01.424 DEBUG:teuthology.orchestra.run.smithi022:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-28T05:00:01.905 INFO:teuthology.provision.fog.smithi022:Node is ready 2023-06-28T05:00:01.905 DEBUG:teuthology.orchestra.run.smithi022:> hostname 2023-06-28T05:00:01.912 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022.front.sepia.ceph.com 2023-06-28T05:00:01.912 DEBUG:teuthology.orchestra.run.smithi022:> grep smithi022.front.sepia.ceph.com /etc/hosts 2023-06-28T05:00:01.919 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2023-06-28T05:00:01.919 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022 2023-06-28T05:00:02.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hosts 2023-06-28T05:00:02.054 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/172.21.15.22/172.21.15.22/g' /etc/hosts 2023-06-28T05:00:02.065 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i -e 's/smithi022.front.sepia.ceph.com/smithi022/g' /etc/hostname 2023-06-28T05:00:02.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname smithi022 2023-06-28T05:00:02.086 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2023-06-28T05:00:02.091 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="Ubuntu" 2023-06-28T05:00:02.091 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-06-28T05:00:02.091 INFO:teuthology.orchestra.run.smithi022.stdout:ID=ubuntu 2023-06-28T05:00:02.092 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE=debian 2023-06-28T05:00:02.092 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-06-28T05:00:02.092 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="20.04" 2023-06-28T05:00:02.092 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://www.ubuntu.com/" 2023-06-28T05:00:02.092 INFO:teuthology.orchestra.run.smithi022.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-06-28T05:00:02.092 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-06-28T05:00:02.092 INFO:teuthology.orchestra.run.smithi022.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-06-28T05:00:02.093 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_CODENAME=focal 2023-06-28T05:00:02.093 INFO:teuthology.orchestra.run.smithi022.stdout:UBUNTU_CODENAME=focal 2023-06-28T05:00:02.093 INFO:teuthology.provision.fog.smithi022:Deploy complete! 2023-06-28T05:00:02.095 DEBUG:teuthology.parallel:result is None 2023-06-28T05:00:55.280 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-06-28T05:00:55.283 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 295, in _wait_for_ready self.remote.run(args=cmd, timeout=600) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 97, in execute self.client.exec_command(self.command, timeout=self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 510, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 919, in open_session return self.open_channel( File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1050, in open_channel raise e File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run ptype, m = self.packetizer.read_message() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message header = self.read_all(self.__block_size_in, check_rekey=True) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all x = self.__socket.recv(n) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv return self._sock.recv(*args) ConnectionResetError: [Errno 104] Connection reset by peer 2023-06-28T05:00:55.284 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T05:00:55.284 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T05:01:00.369 INFO:teuthology.nuke:Checking targets against current locks 2023-06-28T05:01:15.443 DEBUG:teuthology.nuke:shortname: smithi022 2023-06-28T05:01:15.444 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T05:01:30.488 DEBUG:teuthology.nuke:shortname: smithi089 2023-06-28T05:01:30.488 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T05:01:35.498 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_22:53:17-rbd-reef-release-distro-default-smithi/7318862', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 04:53:24.529676', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-28T05:01:45.029 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_22:53:17-rbd-reef-release-distro-default-smithi/7318862', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 04:53:24.532698', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-06-28T05:01:45.033 DEBUG:teuthology.nuke:shortname: smithi111 2023-06-28T05:01:45.034 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T05:01:45.063 INFO:teuthology.orchestra.console:Power off smithi022 2023-06-28T05:01:45.064 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T05:01:45.076 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-06-27_22:53:17-rbd-reef-release-distro-default-smithi/7318862', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 04:53:24.526139', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC63UoZ2hSN4OXU6CYobr4k69dgvAXBaaiTmXZfWKbwYfZjSBci/UKpaEsmwp15Zd5sJaqxjitNRikT3iCcCEWxOWcg74PAng/Zoiy0mRMV/U10cl/3sYCMjJD27suV5R0zICS0WW5K8mKnqV1oB5jYLe0LfE0x2optiSJhy3ny4DV6/k0Gxu5qePqlqYp56lZcwKcUCHYA8FGmGtcfKlHCtljA/qYztm8mqniq5WYjYGxH4m9M2CL6kIVx0/W1oYWr/btRpm+FxfUV+URoEE2yI5y+4bkmz+uX+N7T+VONYv18Pu4HaRvpmIayaMxskIqerOj33w/Uq9ZmAyZ/eHQ2yeocOR2u3RrMXijjyErVC+3vFHzFcj/2IKsFt6kbMJeRGBs8RhmnyeQaLBUFQXaHf8u9wqblHe5RN1Q67s4iYJq0IIvO9uV05uCNE46BsGfVXFM3Ywjo484Q6uIYry2Q2tZhdHebJgVTC9wE0fFK09hz4fS63l3WSletiFWOGis='} 2023-06-28T05:01:45.086 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:45.109 INFO:teuthology.orchestra.console:Power off smithi089 2023-06-28T05:01:45.109 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T05:01:45.130 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:45.146 INFO:teuthology.orchestra.console:Power off smithi111 2023-06-28T05:01:45.147 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T05:01:45.171 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:49.120 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:49.158 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:49.194 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:54.231 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:54.268 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:54.303 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:59.341 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:59.378 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:59.412 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:01:59.590 INFO:teuthology.orchestra.console:Power off for smithi089 completed 2023-06-28T05:01:59.624 INFO:teuthology.orchestra.console:Power off for smithi111 completed 2023-06-28T05:02:03.374 INFO:teuthology.lock.ops:unlocked: smithi089.front.sepia.ceph.com 2023-06-28T05:02:03.382 INFO:teuthology.lock.ops:unlocked: smithi111.front.sepia.ceph.com 2023-06-28T05:02:07.313 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:02:12.424 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:02:17.534 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:02:22.644 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T05:02:22.858 INFO:teuthology.orchestra.console:Power off for smithi022 completed 2023-06-28T05:02:33.020 INFO:teuthology.lock.ops:unlocked: smithi022.front.sepia.ceph.com 2023-06-28T05:02:43.405 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 295, in _wait_for_ready self.remote.run(args=cmd, timeout=600) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 97, in execute self.client.exec_command(self.command, timeout=self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 510, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 919, in open_session return self.open_channel( File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1050, in open_channel raise e File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run ptype, m = self.packetizer.read_message() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message header = self.read_all(self.__block_size_in, check_rekey=True) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all x = self.__socket.recv(n) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv return self._sock.recv(*args) ConnectionResetError: [Errno 104] Connection reset by peer