2023-06-28T13:34:40.620 INFO:root:teuthology version: 0.0.1.dev132+gaa9345a 2023-06-28T13:34:40.621 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T13:34:40.622 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T13:34:59.610 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f93ce061700>, signals=[15]) 2023-06-28T13:34:59.612 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging 2023-06-28T13:34:59.612 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server 2023-06-28T13:34:59.613 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2023-06-28T13:34:59.631 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging 2023-06-28T13:34:59.632 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server 2023-06-28T13:34:59.632 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2023-06-28T13:34:59.634 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /host {"name": "smithi033"} 2023-06-28T13:35:04.655 INFO:teuthology.lock.ops:Start node 'smithi196.front.sepia.ceph.com' reimaging 2023-06-28T13:35:04.655 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: reset os type and version on server 2023-06-28T13:35:04.656 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2023-06-28T13:35:04.657 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /host {"name": "smithi118"} 2023-06-28T13:35:14.671 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /host {"name": "smithi196"} 2023-06-28T13:35:24.688 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-06-28T13:35:24.719 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-06-28T13:35:24.739 DEBUG:teuthology.provision.fog.smithi033:Requesting image smithi_rhel_8.6 (ID 5) 2023-06-28T13:35:24.739 DEBUG:teuthology.provision.fog.smithi033:FOG request: PUT /host/138 {"imageID": 5} 2023-06-28T13:35:29.752 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of rhel 8.6 2023-06-28T13:35:29.753 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:35:29.754 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_rhel_8.6 (ID 5) 2023-06-28T13:35:29.754 DEBUG:teuthology.provision.fog.smithi118:FOG request: PUT /host/223 {"imageID": 5} 2023-06-28T13:35:39.794 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-06-28T13:35:39.911 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /tasktype {"name": "deploy"} 2023-06-28T13:35:39.913 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of rhel 8.6 2023-06-28T13:35:39.913 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-06-28T13:35:40.077 DEBUG:teuthology.provision.fog.smithi033:FOG request: POST /host/138/task {"taskTypeID": 1} 2023-06-28T13:35:40.081 DEBUG:teuthology.provision.fog.smithi196:Requesting image smithi_rhel_8.6 (ID 5) 2023-06-28T13:35:40.081 DEBUG:teuthology.provision.fog.smithi196:FOG request: PUT /host/301 {"imageID": 5} 2023-06-28T13:35:40.183 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of rhel 8.6 2023-06-28T13:35:40.184 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None 2023-06-28T13:35:40.188 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:35:40.198 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /tasktype {"name": "deploy"} 2023-06-28T13:35:40.243 DEBUG:teuthology.provision.fog.smithi118:FOG request: POST /host/223/task {"taskTypeID": 1} 2023-06-28T13:35:44.939 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-06-28T13:35:44.942 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /tasktype {"name": "deploy"} 2023-06-28T13:35:44.974 INFO:teuthology.orchestra.console:Power off smithi033 2023-06-28T13:35:44.975 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T13:35:44.991 DEBUG:teuthology.provision.fog.smithi196:FOG request: POST /host/301/task {"taskTypeID": 1} 2023-06-28T13:35:44.998 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:35:50.041 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2023-06-28T13:35:50.043 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None 2023-06-28T13:35:54.111 INFO:teuthology.orchestra.console:Power on smithi033 2023-06-28T13:35:54.112 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-28T13:35:54.132 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:35:54.147 INFO:teuthology.orchestra.console:Power off smithi118 2023-06-28T13:35:54.148 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T13:35:54.167 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:35:57.045 INFO:teuthology.orchestra.console:Power off for smithi118 completed 2023-06-28T13:35:57.111 INFO:teuthology.orchestra.console:Power off smithi196 2023-06-28T13:35:57.112 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T13:35:57.134 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:35:57.146 INFO:teuthology.orchestra.console:Power on smithi118 2023-06-28T13:35:57.147 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-28T13:35:57.170 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:35:57.257 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2023-06-28T13:35:57.358 INFO:teuthology.orchestra.console:Power on smithi196 2023-06-28T13:35:57.359 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-06-28T13:35:57.379 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:35:58.157 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:36:01.195 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:36:01.402 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:36:01.409 INFO:teuthology.orchestra.console:Power on for smithi118 completed 2023-06-28T13:36:01.510 INFO:teuthology.provision.fog.smithi118:Waiting for deploy to finish 2023-06-28T13:36:01.511 DEBUG:teuthology.provision.fog.smithi118:FOG request: GET /task/active None 2023-06-28T13:36:06.519 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:36:06.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:36:11.648 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:36:11.757 INFO:teuthology.orchestra.console:Power on for smithi033 completed 2023-06-28T13:36:11.858 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2023-06-28T13:36:11.858 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:36:16.865 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-06-28T13:36:20.868 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:36:22.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:36:27.875 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:36:30.928 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-06-28T13:36:31.955 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:36:36.961 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:36:36.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:36:42.079 INFO:teuthology.orchestra.console:Power on for smithi196 completed 2023-06-28T13:36:42.180 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2023-06-28T13:36:42.181 DEBUG:teuthology.provision.fog.smithi196:FOG request: GET /task/active None 2023-06-28T13:36:47.185 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-06-28T13:36:47.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:36:53.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:36:58.193 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:37:03.201 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-06-28T13:37:09.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:37:10.768 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2023-06-28T13:37:12.276 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-06-28T13:37:16.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:37:21.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:37:26.870 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:37:40.208 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2023-06-28T13:37:44.636 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:37:45.328 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2023-06-28T13:37:46.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:37:46.330 ERROR:teuthology.orchestra.connection:Error authenticating with smithi196.front.sepia.ceph.com: Authentication failed. 2023-06-28T13:37:46.331 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-28T13:37:46.332 DEBUG:teuthology.orchestra.run.smithi196:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-28T13:37:46.333 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:47.336 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:48.339 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:49.343 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:50.347 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:51.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:37:56.337 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:56.409 ERROR:teuthology.orchestra.connection:Error authenticating with smithi118.front.sepia.ceph.com: Authentication failed. 2023-06-28T13:37:56.409 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-28T13:37:56.410 DEBUG:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-28T13:37:56.410 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:57.345 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:57.417 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:58.353 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:58.425 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:59.360 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:59.432 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:37:59.772 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:38:04.777 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:04.778 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:05.783 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:05.784 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:06.789 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:06.790 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:07.795 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:07.796 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:08.801 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:08.801 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:09.807 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:09.807 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:10.812 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:10.812 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:11.818 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:11.818 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:12.824 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:12.824 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:13.830 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:13.830 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:14.836 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:14.836 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:15.841 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:15.842 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:16.847 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:16.847 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:17.852 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:17.853 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:18.857 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:18.858 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:19.863 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:19.863 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:19.976 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:38:20.867 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:20.868 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:21.872 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:21.872 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:22.877 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:22.878 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:23.882 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:23.883 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:24.887 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:24.888 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:25.893 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:25.894 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:26.899 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:26.900 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:27.906 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:27.906 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:28.912 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:28.912 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:29.918 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:29.918 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:30.923 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:30.924 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:31.928 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:31.929 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:32.934 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:32.934 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:33.939 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:33.939 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:34.944 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:34.945 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:35.061 DEBUG:teuthology.provision.fog.smithi033:FOG request: GET /task/active None 2023-06-28T13:38:35.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:38:35.950 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:35.951 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:36.956 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:36.956 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:37.962 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:37.962 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:38.968 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:38.969 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:39.974 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:39.974 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:40.979 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:40.980 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:41.986 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:41.986 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:42.992 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:42.992 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:43.998 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:43.998 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:45.003 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:45.004 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:46.009 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:46.010 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:47.015 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:47.015 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:48.021 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:48.021 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:49.027 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:49.027 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:50.033 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:50.033 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:51.038 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:51.039 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:52.044 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:52.045 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:53.050 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:53.051 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:53.680 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-06-28T13:38:54.056 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:54.056 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:55.061 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:55.062 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:56.066 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:56.067 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:57.072 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:57.072 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:58.077 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:58.078 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:59.083 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:59.083 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:38:59.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:39:04.689 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-06-28T13:39:07.760 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2023-06-28T13:39:13.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:40:18.765 WARNING:teuthology.provision.fog.smithi033:timed out 2023-06-28T13:40:24.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T13:40:25.108 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-06-28T13:40:25.108 DEBUG:teuthology.orchestra.run.smithi033:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-06-28T13:40:25.418 INFO:teuthology.provision.fog.smithi033:Node is ready 2023-06-28T13:40:25.419 DEBUG:teuthology.orchestra.run.smithi033:> hostname 2023-06-28T13:40:25.437 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com 2023-06-28T13:40:25.437 DEBUG:teuthology.orchestra.run.smithi033:> grep smithi033.front.sepia.ceph.com /etc/hosts 2023-06-28T13:40:25.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033 2023-06-28T13:40:25.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2023-06-28T13:40:25.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hosts 2023-06-28T13:40:25.640 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/172.21.15.33/172.21.15.33/g' /etc/hosts 2023-06-28T13:40:25.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hostname 2023-06-28T13:40:25.778 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2023-06-28T13:40:25.846 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2023-06-28T13:40:25.902 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Red Hat Enterprise Linux" 2023-06-28T13:40:25.902 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8.6 (Ootpa)" 2023-06-28T13:40:25.903 INFO:teuthology.orchestra.run.smithi033.stdout:ID="rhel" 2023-06-28T13:40:25.903 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="fedora" 2023-06-28T13:40:25.903 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8.6" 2023-06-28T13:40:25.903 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8" 2023-06-28T13:40:25.903 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-28T13:40:25.903 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2023-06-28T13:40:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-28T13:40:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.redhat.com/" 2023-06-28T13:40:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-28T13:40:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-28T13:40:25.904 INFO:teuthology.orchestra.run.smithi033.stdout: 2023-06-28T13:40:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-28T13:40:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-28T13:40:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-28T13:40:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-28T13:40:25.906 INFO:teuthology.provision.fog.smithi033:Deploy complete! 2023-06-28T13:40:25.907 DEBUG:teuthology.parallel:result is None 2023-06-28T13:40:52.464 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-06-28T13:40:52.467 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 295, in _wait_for_ready self.remote.run(args=cmd, timeout=600) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 97, in execute self.client.exec_command(self.command, timeout=self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 510, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 919, in open_session return self.open_channel( File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1050, in open_channel raise e File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run ptype, m = self.packetizer.read_message() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message header = self.read_all(self.__block_size_in, check_rekey=True) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all x = self.__socket.recv(n) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv return self._sock.recv(*args) ConnectionResetError: [Errno 104] Connection reset by peer 2023-06-28T13:40:52.467 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T13:40:52.469 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-06-28T13:40:52.548 INFO:teuthology.nuke:Checking targets against current locks 2023-06-28T13:40:52.606 DEBUG:teuthology.nuke:shortname: smithi033 2023-06-28T13:40:52.606 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T13:40:52.630 DEBUG:teuthology.nuke:shortname: smithi118 2023-06-28T13:40:52.630 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T13:40:52.632 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319811', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 13:34:25.297397', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDVacGiMKfbhq7SXBkEWg8WiT3k/DxsgdZKeMemfUDR7Fm1Ttd5U1vXW6d0Kh+ehLmtPoXE2qZxcrcla4EjaillzGRBa/3YtggMz9WRucB6zsfm/Yd7Q9WMWuptvrG9Y9Ap3PcPNRXKyzj5i1TeAWVJ/MuBFAkMbkUtjnW0rXk2xYYlrJ4Mbr4qS9NiXFxke9cFPfO+PfFjJu62glsJ7PpIcwf4YW1/90FuhHckR84BUVlgdIE3H42dozWv8b/0TA269BLLBq5JD+0dw3DJB4469gRFXeZjGKQubxS2cbSms3R4Bp7WFHkKXUS8EwLTxqr3vhSTy6pHiX2zkhzTZO/sDlIYQbnjedllesx2iSFZUFMNVK+6osHg/rUAbvA4yX2RgRU6YUSwgKE0vvQKq/NE8TGs+aBL2vy2F4UVejKLDMFRUKW8kGGdJCqtnGdstIDHYCiuW/rjI+LansiCX91gHUBzBTP3SvAEaaUHeeRAsxoaoqfH1YUrQMVvCmWjXSk='} 2023-06-28T13:40:52.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi118.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319811', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 13:34:25.293237', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCcerlxHmnLZyf5eV3bUPt5xj9cPtI7gGIFsSwKrDAu3rSE0GlE40qW1/5S4HNjNW0FzCu0YxiJbqa478OuC0Fgrk7fQONALYwia4RZ6hJQVD0RCrbIEV8ttdJGjXj5XhJerwEPd5GUWS32+rubMjlWYCWVwpBnNw2ZHz5syJlzj2eaFI39UoJY24VggU/lJC7Q9EFRUAtSsBRzC/GXy5EKb4rQ35iSDnyf71yEg/EfTw6Cfu/oq8OD2Rpdrlyser7j+R/4hlflLCllDSUN0wTKpCc/BgXRheqlz5yMJe4SavnLZuL5Q7is99jD5egkhdifFiMtKYLz3z2L5elQjN5R3SKpsmrPp4Fx8JKJMnKE4sJfMW+8OlUUBFHqeMmWrq/NjJoA5mPmSBBrHQG6lDoOfbCL8Y2qrauIYOSLynHOf1ujPO6wXnRXjtroOByN29DjdV/iDpJHrAxi8AJAXn4+fOxs9Oj5qGb4GAgBdeCFa8S6sNjcpZ/L4EKc8WO8f8s='} 2023-06-28T13:40:52.662 DEBUG:teuthology.nuke:shortname: smithi196 2023-06-28T13:40:52.662 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T13:40:52.681 INFO:teuthology.orchestra.console:Power off smithi033 2023-06-28T13:40:52.682 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T13:40:52.692 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319811', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 13:34:25.295588', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnwJ5SiXhyStrVwkVqkQD3HvNTlboXk9IBqU5EN4YxJX/SEnvHftVrwf2RD1iC6swJknc9aDs3iyjfIniId6YHmqV38oi9ua5U67qqn7ClJr+pebkkFJgffXEYKKhlois46R3JRJi9R6ZcciF+Jtna/l49rYiIF9KGtRNMFeyXHDYIps8OChQntoZ8zfhGF7nL89v3l3ZM0w0D1AtbdYZhWdaJkAxJAgqxMLiWrD8WBlKlgTd3ZVcp8OQh+NfbZMuWN6UIHD+okqMZ4+XWqjwL3m2kc0hEBQBxylQ5rUMbbPTdJGU5+da2OBUt1NqddrMzxSR4ytL30oQbCpmKusjJL4Gz2jDTjXRt3TnnQGq4XhN2OIIPdFLoPH0fVJaZfuqJj8+MF0PYqRv//EkP6bqijxEzEqxrapgW1200JwEw7L/hSGh9ScLQjNmyPOp2RJxqH9aebOMLZwWcEbgqKisWRZ1SVU/fN8eqeIDAbJerCxHMrgzT9uH/+C5ZmywcVL0='} 2023-06-28T13:40:52.704 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:40:52.726 INFO:teuthology.orchestra.console:Power off smithi118 2023-06-28T13:40:52.726 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T13:40:52.748 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:40:52.763 INFO:teuthology.orchestra.console:Power off smithi196 2023-06-28T13:40:52.763 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T13:40:52.786 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:40:56.737 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:40:56.775 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:40:56.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:01.847 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:01.886 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:01.919 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:02.099 INFO:teuthology.orchestra.console:Power off for smithi118 completed 2023-06-28T13:41:12.234 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:12.242 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:12.276 INFO:teuthology.lock.ops:unlocked: smithi118.front.sepia.ceph.com 2023-06-28T13:41:17.345 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:17.352 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:22.455 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:22.465 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:27.569 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:27.579 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:27.781 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2023-06-28T13:41:37.925 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:37.957 INFO:teuthology.lock.ops:unlocked: smithi033.front.sepia.ceph.com 2023-06-28T13:41:43.035 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:48.147 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:53.257 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:58.368 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T13:41:58.580 INFO:teuthology.orchestra.console:Power off for smithi196 completed 2023-06-28T13:42:08.751 INFO:teuthology.lock.ops:unlocked: smithi196.front.sepia.ceph.com 2023-06-28T13:42:14.298 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/provision/fog.py", line 295, in _wait_for_ready self.remote.run(args=cmd, timeout=600) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 450, in run r.execute() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 97, in execute self.client.exec_command(self.command, timeout=self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/client.py", line 510, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 919, in open_session return self.open_channel( File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 1050, in open_channel raise e File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/transport.py", line 2109, in run ptype, m = self.packetizer.read_message() File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 459, in read_message header = self.read_all(self.__block_size_in, check_rekey=True) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/paramiko/packet.py", line 301, in read_all x = self.__socket.recv(n) File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/virtualenv/lib/python3.8/site-packages/gevent/_socketcommon.py", line 657, in recv return self._sock.recv(*args) ConnectionResetError: [Errno 104] Connection reset by peer