2023-07-27T05:19:04.750 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-27T05:19:04.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T05:19:04.752 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T05:19:20.158 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f99062b2310>, signals=[15]) 2023-07-27T05:19:20.160 INFO:teuthology.lock.ops:Start node 'smithi002.front.sepia.ceph.com' reimaging 2023-07-27T05:19:20.160 INFO:teuthology.lock.ops:Updating [smithi002.front.sepia.ceph.com]: reset os type and version on server 2023-07-27T05:19:20.161 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2023-07-27T05:19:26.836 INFO:teuthology.lock.ops:Start node 'smithi027.front.sepia.ceph.com' reimaging 2023-07-27T05:19:26.836 INFO:teuthology.lock.ops:Updating [smithi027.front.sepia.ceph.com]: reset os type and version on server 2023-07-27T05:19:26.836 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-07-27T05:19:26.838 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /host {"name": "smithi002"} 2023-07-27T05:19:36.868 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /host {"name": "smithi027"} 2023-07-27T05:19:42.208 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-27T05:19:50.532 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-27T05:19:50.548 DEBUG:teuthology.provision.fog.smithi002:Requesting image smithi_centos_8.stream (ID 3) 2023-07-27T05:19:50.549 DEBUG:teuthology.provision.fog.smithi002:FOG request: PUT /host/108 {"imageID": 3} 2023-07-27T05:19:50.581 DEBUG:teuthology.provision.fog.smithi027:Requesting image smithi_centos_8.stream (ID 3) 2023-07-27T05:19:50.581 DEBUG:teuthology.provision.fog.smithi027:FOG request: PUT /host/132 {"imageID": 3} 2023-07-27T05:19:50.592 INFO:teuthology.provision.fog.smithi002:Scheduling deploy of centos 8.stream 2023-07-27T05:19:50.592 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-07-27T05:19:50.625 INFO:teuthology.provision.fog.smithi027:Scheduling deploy of centos 8.stream 2023-07-27T05:19:50.625 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-27T05:19:50.674 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /tasktype {"name": "deploy"} 2023-07-27T05:19:50.697 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /tasktype {"name": "deploy"} 2023-07-27T05:19:50.707 DEBUG:teuthology.provision.fog.smithi002:FOG request: POST /host/108/task {"taskTypeID": 1} 2023-07-27T05:19:50.735 DEBUG:teuthology.provision.fog.smithi027:FOG request: POST /host/132/task {"taskTypeID": 1} 2023-07-27T05:19:50.771 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-07-27T05:19:50.795 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-27T05:20:00.839 INFO:teuthology.orchestra.console:Power off smithi002 2023-07-27T05:20:00.841 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T05:20:05.909 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T05:20:05.910 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:05.941 INFO:teuthology.orchestra.console:Power off smithi027 2023-07-27T05:20:05.942 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T05:20:05.964 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T05:20:05.965 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:05.988 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T05:20:06.920 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:09.992 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:10.922 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:11.101 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:12.032 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:15.102 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:15.212 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T05:20:15.313 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2023-07-27T05:20:15.414 INFO:teuthology.orchestra.console:Power on smithi027 2023-07-27T05:20:15.415 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-27T05:20:16.034 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:17.144 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:20.444 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-27T05:20:20.444 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:21.145 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:21.255 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T05:20:21.356 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2023-07-27T05:20:21.452 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:21.458 INFO:teuthology.orchestra.console:Power on smithi002 2023-07-27T05:20:21.458 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-27T05:20:25.456 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:26.482 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-27T05:20:26.483 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:26.566 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:27.492 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:30.567 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:31.494 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:31.603 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T05:20:31.677 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:31.704 INFO:teuthology.orchestra.console:Power on for smithi002 completed 2023-07-27T05:20:31.805 INFO:teuthology.provision.fog.smithi002:Waiting for deploy to finish 2023-07-27T05:20:31.805 DEBUG:teuthology.provision.fog.smithi002:FOG request: GET /task/active None 2023-07-27T05:20:36.813 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:36.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:20:37.924 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:39.984 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-07-27T05:20:41.925 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:43.034 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T05:20:45.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:20:47.036 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:20:47.147 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T05:20:47.248 INFO:teuthology.orchestra.console:Power on for smithi027 completed 2023-07-27T05:20:47.349 INFO:teuthology.provision.fog.smithi027:Waiting for deploy to finish 2023-07-27T05:20:47.350 DEBUG:teuthology.provision.fog.smithi027:FOG request: GET /task/active None 2023-07-27T05:20:52.357 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-07-27T05:20:52.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:20:58.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:21:00.528 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-27T05:21:01.424 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-07-27T05:21:06.532 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:21:11.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:21:16.541 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-27T05:21:19.600 WARNING:teuthology.provision.fog.smithi002:[Errno None] Unable to connect to port 22 on 172.21.15.2 2023-07-27T05:21:22.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:21:27.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:21:32.555 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-27T05:21:38.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:21:41.620 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-27T05:21:47.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:21:55.696 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-27T05:22:01.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:22:04.752 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-27T05:22:10.756 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:22:20.912 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-27T05:22:26.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:22:29.968 WARNING:teuthology.provision.fog.smithi027:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-07-27T05:22:32.557 WARNING:teuthology.provision.fog.smithi002:timed out 2023-07-27T05:22:35.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:22:40.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:22:46.058 ERROR:teuthology.orchestra.connection:Error authenticating with smithi027.front.sepia.ceph.com: Authentication failed. 2023-07-27T05:22:46.060 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T05:22:46.061 DEBUG:teuthology.orchestra.run.smithi027:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T05:22:46.062 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:46.065 ERROR:teuthology.orchestra.connection:Error authenticating with smithi002.front.sepia.ceph.com: Authentication failed. 2023-07-27T05:22:46.065 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T05:22:46.066 DEBUG:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T05:22:46.066 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:47.072 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:47.075 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:48.081 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:48.083 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:49.090 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:49.091 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:50.100 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:50.101 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:51.107 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:51.108 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:52.114 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:52.115 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:53.121 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:53.122 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:54.130 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:54.131 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:55.137 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:55.138 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:56.145 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:56.145 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:57.151 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:57.151 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:58.160 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:58.160 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:59.166 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:22:59.167 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:00.174 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:00.175 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:01.181 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:01.182 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:02.189 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:02.190 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:03.196 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:03.197 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:04.203 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:04.203 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:05.210 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:05.210 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:06.217 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:06.217 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:07.224 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:07.225 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:08.233 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:08.234 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:09.241 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:09.241 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:10.247 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:10.248 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:11.254 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:11.254 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:12.261 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:12.262 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:13.268 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:13.269 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:14.275 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:14.276 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:15.282 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:15.282 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:16.289 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:16.290 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:17.297 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:17.298 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:18.305 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:18.306 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:19.313 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:19.314 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:20.320 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:20.320 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:21.329 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:21.329 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:22.336 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:22.336 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:23.343 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:23.345 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:24.351 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:24.351 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:25.358 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:25.358 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:26.365 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:26.365 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:27.371 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:27.372 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:28.378 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:28.378 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:29.385 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:29.385 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:30.392 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:30.392 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:31.398 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:31.399 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:32.404 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:32.405 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:33.411 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:33.411 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:34.418 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:34.418 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:35.424 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:35.425 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:36.431 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:36.432 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:37.438 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:37.439 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:38.446 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:38.447 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:39.453 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:39.454 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:40.461 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:40.461 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:41.467 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:41.467 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:42.473 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:42.474 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:43.479 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:43.480 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:44.486 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:44.487 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:45.493 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:45.493 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:46.500 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:46.500 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:47.506 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:47.507 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:48.513 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:48.514 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:49.523 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:49.523 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:50.529 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:50.530 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:51.537 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:51.537 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:52.544 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:52.544 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:53.550 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:53.551 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:54.557 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:54.557 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:55.563 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:55.563 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:56.569 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:56.570 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:57.576 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:57.577 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:58.583 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:58.583 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:59.589 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:23:59.590 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:00.597 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:00.598 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:01.604 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:01.605 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:02.612 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:02.613 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:03.618 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:03.619 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:04.626 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:05.633 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:06.640 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:07.648 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:08.656 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:09.664 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:10.672 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:11.679 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:12.688 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:13.697 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:14.704 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:15.712 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:16.720 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:17.729 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:18.737 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:19.746 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:20.754 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:21.762 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:22.770 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:23.779 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:24.786 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:25.796 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:26.805 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:27.812 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:28.821 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:29.832 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:30.840 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:31.849 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:32.857 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:33.865 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:34.873 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:35.881 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:36.889 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:37.898 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:38.906 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:39.914 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:40.924 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:24:40.988 ERROR:teuthology.provision.fog: on smithi027 2023-07-27T05:24:49.989 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-27T05:24:49.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:24:55.065 ERROR:teuthology.orchestra.connection:Error authenticating with smithi027.front.sepia.ceph.com: Authentication failed. 2023-07-27T05:24:55.065 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-07-27T05:24:55.066 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:25:51.344 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-27T05:25:51.345 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi002 2023-07-27T05:26:00.347 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-07-27T05:26:00.348 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:26:00.602 DEBUG:teuthology.orchestra.run.smithi002:> true 2023-07-27T05:26:00.878 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-07-27T05:26:00.878 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T05:26:00.879 DEBUG:teuthology.orchestra.run.smithi002:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T05:26:00.898 INFO:teuthology.provision.fog.smithi002:Node is ready 2023-07-27T05:26:00.899 DEBUG:teuthology.orchestra.run.smithi002:> hostname 2023-07-27T05:26:00.961 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002.front.sepia.ceph.com 2023-07-27T05:26:00.962 DEBUG:teuthology.orchestra.run.smithi002:> grep smithi002.front.sepia.ceph.com /etc/hosts 2023-07-27T05:26:01.025 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2 smithi002.front.sepia.ceph.com smithi002 2023-07-27T05:26:01.025 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2023-07-27T05:26:01.140 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hosts 2023-07-27T05:26:01.170 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/172.21.15.2/172.21.15.2/g' /etc/hosts 2023-07-27T05:26:01.244 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i -e 's/smithi002.front.sepia.ceph.com/smithi002/g' /etc/hostname 2023-07-27T05:26:01.316 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname smithi002 2023-07-27T05:26:01.387 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2023-07-27T05:26:01.449 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2023-07-27T05:26:01.450 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8" 2023-07-27T05:26:01.450 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2023-07-27T05:26:01.450 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2023-07-27T05:26:01.450 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8" 2023-07-27T05:26:01.450 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2023-07-27T05:26:01.450 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 8" 2023-07-27T05:26:01.451 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2023-07-27T05:26:01.451 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-07-27T05:26:01.451 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2023-07-27T05:26:01.451 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T05:26:01.451 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T05:26:01.451 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-27T05:26:01.452 INFO:teuthology.provision.fog.smithi002:Deploy complete! 2023-07-27T05:26:01.453 DEBUG:teuthology.parallel:result is None 2023-07-27T05:27:04.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-27T05:27:04.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:27:04.118 ERROR:teuthology.orchestra.connection:Error authenticating with smithi027.front.sepia.ceph.com: Authentication failed. 2023-07-27T05:27:04.118 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-07-27T05:27:04.119 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:29:16.012 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-27T05:29:16.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:29:16.129 ERROR:teuthology.orchestra.connection:Error authenticating with smithi027.front.sepia.ceph.com: Authentication failed. 2023-07-27T05:29:16.129 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-07-27T05:29:16.132 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:31:19.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-07-27T05:31:19.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T05:31:19.101 ERROR:teuthology.orchestra.connection:Error authenticating with smithi027.front.sepia.ceph.com: Authentication failed. 2023-07-27T05:31:19.102 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-07-27T05:31:19.103 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-27T05:33:19.036 WARNING:teuthology.contextutil:'reconnect to {self.shortname}' reached maximum tries (5) after waiting for 30 seconds 2023-07-27T05:33:19.038 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 299, in _wait_for_ready self.remote.run(args=cmd, timeout=600) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi027 2023-07-27T05:33:19.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T05:33:19.039 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T05:33:24.129 INFO:teuthology.nuke:Checking targets against current locks 2023-07-27T05:33:43.998 DEBUG:teuthology.nuke:shortname: smithi002 2023-07-27T05:33:43.999 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-27T05:33:47.199 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-26_15:54:22-rados-wip-yuri6-testing-2023-07-24-0819-pacific-distro-default-smithi/7353579', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-27 05:18:58.696910', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIVSeKTV3o+KX2tAfclrPrsDddKfJh2QtVQKuvr1DE8nCske8V2aGNqhmZvwim74U1GNPWFbqeCLgFZZA6b2dK75vKaYLp34zCK/0RKen5L21zX79lTtnBbtv96LXkF1/tGGvJBVaondOyFHvpewun/Yn42LXtDL34i37mUw8yeOIiw3bS1ZNCNl63jfcis40D9lecVpaMe9CB9Pvt59U0PHoLi9vbhl4CocZQx6Vgc+D6Himdl6exzZ1qoh0d9SXCG2EzuYyzOpeuw919xPLwDMN7KowkRCkrz6/cxzlcoxQObLbcuE/Twq5Bfj3kPLcM0XDkXGdmLvacIMlkdH9wh5bnGC9H8/nfUSaAkSQ5G1Bh6p9SKniLCvROkfqFVhOSYyavJ8IxGN8Wd9mXvpRVa08+M+tdsoaQMhx0kuA8GEYc9WPdN65ywNtcu6Mh1a452ysS5mexelLgGpWkpHk/ke9ctbH8gxYUwzytQisb9rF5aTu0yxSgtW/3iHYSr2s='} 2023-07-27T05:33:47.216 DEBUG:teuthology.nuke:shortname: smithi027 2023-07-27T05:33:47.217 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-27T05:33:47.256 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-26_15:54:22-rados-wip-yuri6-testing-2023-07-24-0819-pacific-distro-default-smithi/7353579', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-27 05:18:58.699238', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYE04XL/CxA7HH8K/pNKMGzfJUv9z55ReRr4ya/zG/CK42gJVEQ6f56X8NU0/6Zvf6l/8r1/Xaow0A3GjXSx1lJvuF3/LITg74sGgY9nA5M2/itpCf+lCccb+I7OkBKO4BnWE80Mia8GhQi5RlOa6YIyzo0dW8g/xCbzVCWafO4NSSxR+mo2GkPA0SbUgTOgrYP01Wd2ECusk3xKgsssQz8s/wccyPN/Jmgic+t+jreIitbzLs+siQQuM1W65+NqqP+WGl/Nvu71lEc0LHx9s2oUUjufDwu3ZjN2ou4EvVn0qnlF1p9nyRvzbt2r8UPpzeYoZDPOBeZSLGQr+3Rth0HRZG/3hhiUFdUlTiJ3Nn8WGS/QGzMk7+/RFC2167bx0aTVMcyNNuQ1oIs6N8XxTncZs7xy5TcFB9/NMsfzkBc1tP4SZetx4gxGB2H1jvrtjcK0O/laQwNGNOaoHt/sZIMTPEE41eFQRd+MbvboRegNCwQHRH+FL2Fdjms2VrJzk='} 2023-07-27T05:33:47.282 INFO:teuthology.orchestra.console:Power off smithi002 2023-07-27T05:33:47.283 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T05:33:47.306 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T05:33:47.307 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:33:47.326 INFO:teuthology.orchestra.console:Power off smithi027 2023-07-27T05:33:47.327 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T05:33:47.336 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T05:33:47.348 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T05:33:47.348 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:33:47.369 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T05:33:51.337 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:33:51.370 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:33:51.448 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T05:33:51.481 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T05:33:55.453 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:33:55.482 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T05:33:55.562 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T05:33:55.592 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T05:33:55.664 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2023-07-27T05:33:55.693 INFO:teuthology.orchestra.console:Power off for smithi027 completed 2023-07-27T05:33:55.824 INFO:teuthology.lock.ops:unlocked: smithi002.front.sepia.ceph.com 2023-07-27T05:33:55.842 INFO:teuthology.lock.ops:unlocked: smithi027.front.sepia.ceph.com 2023-07-27T05:33:56.094 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 90, in create self._wait_for_ready() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 299, in _wait_for_ready self.remote.run(args=cmd, timeout=600) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/orchestra/remote.py", line 521, in run raise Exception(f'Cannot connect to remote host {self.shortname}') Exception: Cannot connect to remote host smithi027