2023-07-25T20:00:01.223 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-25T20:00:01.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-25T20:00:01.225 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-25T20:00:01.412 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa8d70f71f0>, signals=[15]) 2023-07-25T20:00:01.414 INFO:teuthology.lock.ops:Start node 'smithi132.front.sepia.ceph.com' reimaging 2023-07-25T20:00:01.415 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: reset os type and version on server 2023-07-25T20:00:01.415 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-07-25T20:00:01.436 INFO:teuthology.lock.ops:Start node 'smithi161.front.sepia.ceph.com' reimaging 2023-07-25T20:00:01.437 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: reset os type and version on server 2023-07-25T20:00:01.437 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2023-07-25T20:00:01.438 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /host {"name": "smithi132"} 2023-07-25T20:00:16.467 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /host {"name": "smithi161"} 2023-07-25T20:00:25.084 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-25T20:00:35.120 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-07-25T20:00:40.126 DEBUG:teuthology.provision.fog.smithi132:Requesting image smithi_centos_8.stream (ID 3) 2023-07-25T20:00:40.126 DEBUG:teuthology.provision.fog.smithi132:FOG request: PUT /host/237 {"imageID": 3} 2023-07-25T20:00:45.181 DEBUG:teuthology.provision.fog.smithi161:Requesting image smithi_centos_8.stream (ID 3) 2023-07-25T20:00:45.182 DEBUG:teuthology.provision.fog.smithi161:FOG request: PUT /host/266 {"imageID": 3} 2023-07-25T20:00:45.200 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of centos 8.stream 2023-07-25T20:00:45.200 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-25T20:00:45.314 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of centos 8.stream 2023-07-25T20:00:45.314 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /task/active None 2023-07-25T20:00:45.339 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /tasktype {"name": "deploy"} 2023-07-25T20:00:45.426 DEBUG:teuthology.provision.fog.smithi132:FOG request: POST /host/237/task {"taskTypeID": 1} 2023-07-25T20:00:45.447 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /tasktype {"name": "deploy"} 2023-07-25T20:00:45.500 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-25T20:00:45.505 DEBUG:teuthology.provision.fog.smithi161:FOG request: POST /host/266/task {"taskTypeID": 1} 2023-07-25T20:00:45.573 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /task/active None 2023-07-25T20:00:55.648 INFO:teuthology.orchestra.console:Power off smithi132 2023-07-25T20:00:55.649 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-25T20:00:55.664 INFO:teuthology.orchestra.console:Power off smithi161 2023-07-25T20:00:55.664 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-25T20:01:00.675 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-25T20:01:00.676 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:00.691 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-25T20:01:00.692 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:01.686 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:01.701 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:05.688 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:05.702 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:06.799 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:06.813 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:10.800 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:10.814 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:11.915 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:11.925 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:15.916 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:15.926 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:17.028 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:17.039 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:21.035 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:21.048 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:22.152 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:22.160 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:26.154 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:26.163 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:27.266 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:27.276 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:31.267 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:31.278 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:32.379 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:32.391 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:36.382 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:36.392 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:37.493 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:37.503 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:41.495 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:41.504 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:41.607 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T20:01:41.709 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2023-07-25T20:01:41.810 INFO:teuthology.orchestra.console:Power on smithi132 2023-07-25T20:01:41.810 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-25T20:01:41.835 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-25T20:01:41.836 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:41.860 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T20:01:42.616 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:45.864 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:46.618 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:46.975 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:47.728 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:01:50.976 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:51.087 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T20:01:51.188 INFO:teuthology.orchestra.console:Power on for smithi132 completed 2023-07-25T20:01:51.289 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2023-07-25T20:01:51.290 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-25T20:01:56.297 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:01:57.408 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:02:01.409 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:02:02.520 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:02:06.524 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:02:07.635 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:02:11.388 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-25T20:02:16.393 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:02:17.504 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:02:21.508 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:02:22.619 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:02:22.620 WARNING:teuthology.contextutil:'wait for power off' reached maximum tries (16) after waiting for 60.0 seconds 2023-07-25T20:02:22.620 ERROR:teuthology.orchestra.console:Failed to power off smithi161 2023-07-25T20:02:22.620 INFO:teuthology.orchestra.console:Power on smithi161 2023-07-25T20:02:22.621 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-25T20:02:26.476 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-25T20:02:26.476 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:02:26.500 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T20:02:30.504 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:02:30.615 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T20:02:30.716 INFO:teuthology.orchestra.console:Power on for smithi161 completed 2023-07-25T20:02:30.817 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish 2023-07-25T20:02:30.817 DEBUG:teuthology.provision.fog.smithi161:FOG request: GET /task/active None 2023-07-25T20:02:35.820 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-25T20:02:40.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:02:49.012 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-07-25T20:02:55.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:03:00.021 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-25T20:03:05.024 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-07-25T20:03:05.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:03:08.144 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-25T20:03:11.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:03:16.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:03:21.038 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-07-25T20:03:22.342 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-25T20:03:27.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:03:32.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:03:37.049 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-07-25T20:03:37.169 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: Authentication failed. 2023-07-25T20:03:37.172 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-25T20:03:37.173 DEBUG:teuthology.orchestra.run.smithi132:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-25T20:03:37.174 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:38.185 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:39.195 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:40.201 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:41.210 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:42.215 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:43.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:03:48.054 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:49.059 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:50.063 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:51.069 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:52.076 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:53.083 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:54.092 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:55.098 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:56.107 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:57.115 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:58.122 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:03:59.130 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:00.138 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:01.152 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:02.161 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:03.169 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:04.178 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:05.185 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:06.191 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:07.197 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:08.203 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:09.211 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:10.217 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:11.226 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:12.232 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:13.239 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:14.246 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:15.252 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:16.259 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:17.265 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:18.272 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:19.281 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:20.289 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:21.296 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:22.305 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:23.313 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:24.331 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:25.343 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:26.353 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:27.364 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:28.370 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:29.380 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:30.386 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:31.393 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:32.400 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:33.410 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:34.421 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:35.430 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:36.436 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:37.441 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:38.448 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:39.455 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:40.462 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:41.471 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:42.478 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:43.484 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:44.490 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:45.501 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:46.508 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:47.514 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:48.054 WARNING:teuthology.provision.fog.smithi161:timed out 2023-07-25T20:04:48.519 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:49.526 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:50.533 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:51.541 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:52.546 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:53.554 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:54.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:04:54.260 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-25T20:04:54.261 DEBUG:teuthology.orchestra.run.smithi161:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-25T20:04:54.561 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:04:54.878 INFO:teuthology.provision.fog.smithi161:Node is ready 2023-07-25T20:04:54.879 DEBUG:teuthology.orchestra.run.smithi161:> hostname 2023-07-25T20:04:54.885 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com 2023-07-25T20:04:54.886 DEBUG:teuthology.orchestra.run.smithi161:> grep smithi161.front.sepia.ceph.com /etc/hosts 2023-07-25T20:04:54.893 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161 2023-07-25T20:04:54.893 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161 2023-07-25T20:04:55.028 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hosts 2023-07-25T20:04:55.046 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/172.21.15.161/172.21.15.161/g' /etc/hosts 2023-07-25T20:04:55.056 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hostname 2023-07-25T20:04:55.074 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161 2023-07-25T20:04:55.128 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2023-07-25T20:04:55.135 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Ubuntu" 2023-07-25T20:04:55.136 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-07-25T20:04:55.136 INFO:teuthology.orchestra.run.smithi161.stdout:ID=ubuntu 2023-07-25T20:04:55.136 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE=debian 2023-07-25T20:04:55.137 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-07-25T20:04:55.137 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="20.04" 2023-07-25T20:04:55.137 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.ubuntu.com/" 2023-07-25T20:04:55.138 INFO:teuthology.orchestra.run.smithi161.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-07-25T20:04:55.138 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-07-25T20:04:55.138 INFO:teuthology.orchestra.run.smithi161.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-07-25T20:04:55.138 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_CODENAME=focal 2023-07-25T20:04:55.139 INFO:teuthology.orchestra.run.smithi161.stdout:UBUNTU_CODENAME=focal 2023-07-25T20:04:55.143 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi161's OS to be centos 8 but found ubuntu 20.04 2023-07-25T20:04:55.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-25T20:04:55.144 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-25T20:05:00.149 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:05:00.233 INFO:teuthology.nuke:Checking targets against current locks 2023-07-25T20:05:05.236 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:05:15.274 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:05:20.306 DEBUG:teuthology.nuke:shortname: smithi132 2023-07-25T20:05:20.307 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-25T20:05:25.312 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:05:25.336 DEBUG:teuthology.nuke:shortname: smithi161 2023-07-25T20:05:25.336 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-25T20:05:30.342 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-07-25_19:56:53-orch:cephadm-wip-guits-testing-2-2023-07-25-0836-distro-default-smithi/7351891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-25 20:00:00.334866', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXGBF5ykefDfSpJG4KdQVex6fz9AC9Sf+SlrxkFDorCdNK8ASUTApHrInPrLgZ09rN9tqSXaF5FULZXCPzvW/if15YGXJtOLaatyPjyJoyTVtaW5IeeNorqkjODaD2FgoNqAyseHSglRd30dtzXiEyMxO4igF/ISnsGmCIVWGJ/Qxqq8rPOr0iezCo1+/U/GJobwH6JwtsE2uyM784vuBif7GQiPy+RR+AtO++lSOLCSHQhlgb0yBWvundrTZ7KxTTckbRBgCXXxTwoM6cm6ADIJC2JMHxFGqHShcIhJWw2NGSD6kIC6hj17zG3rdix2dfQP4veZ7mB51t7obh1AL/C3JUPeZQH7i0jwgZTPRFI3zJKRMi7gxpS73bd9C791XjjIDwSXaA9Xak8q0+ROL0PY42u9dNE/EkwltFEAB/aNxBVyFqDR6vC8S5/L17H/GMqsMAjeS1RxCHRyu/3LW7V+wv1NGLSZcZG/TyrAUz7z24JU3bxybY4Hik0IIwktE='} 2023-07-25T20:05:35.348 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:05:35.373 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2023-07-25_19:56:53-orch:cephadm-wip-guits-testing-2-2023-07-25-0836-distro-default-smithi/7351891', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-25 20:00:00.333426', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwINnmi+N3LgR9ET61cABYBnDJDAAQE1klaPINljTOvitiIM3/5ICam+qurn41RkfyLPz4uwE1yDyHeI7QBKChMImagaJPfltOSsFsTn67ifp6uunrBJblDzycEd7/lJfdMwBn8dApfOw5SSnVdpzpaKWGNpPqshrinViULNhTz2PMAkza/R7qM6v4qqKKxQx/yVi8/Lt03nxO672k5WH/Wkz7KHX4nwyeTelksSojH9SBt4yR4WpLbMjuFX33Ag14KBWr9YQ30L5/h7VtBLQL+2c7sq3Bnll5zj8H+sIa0hOHmwR5sxs4cVs6Eb4Cblr6xTs0Q9vQefWDDYNL4qRpuj1NO+bW1xGcALK9TujCJ0/fdcMLQ0/r4QukVxSE1j6frwWwo/u8lxTTp/wbNVU14w+37ppSvzcPbOD5FNnCu4PMHgrtqp02ggncjqMYpHvxXoQ53JPn84XfXX04vWf5emRMQ76T0NvWW0jYtj7x7ex7l/iEGUGHVnaOmjLqij0='} 2023-07-25T20:05:40.383 ERROR:teuthology.provision.fog: on smithi132 2023-07-25T20:05:50.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-07-25T20:05:50.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-25T20:05:55.431 INFO:teuthology.orchestra.console:Power off smithi132 2023-07-25T20:05:55.432 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-25T20:05:55.471 INFO:teuthology.orchestra.console:Power off smithi161 2023-07-25T20:05:55.472 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-25T20:05:55.495 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-25T20:05:55.496 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:05:55.518 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T20:05:55.519 ERROR:teuthology.orchestra.connection:Error authenticating with smithi132.front.sepia.ceph.com: Authentication failed. 2023-07-25T20:05:55.519 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-07-25T20:05:55.520 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-25T20:05:59.520 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:05:59.630 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T20:06:03.632 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:03.745 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-25T20:06:07.746 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:07.857 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T20:06:07.958 INFO:teuthology.orchestra.console:Power off for smithi161 completed 2023-07-25T20:06:12.747 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-25T20:06:12.748 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:15.890 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:06:15.925 INFO:teuthology.lock.ops:unlocked: smithi161.front.sepia.ceph.com 2023-07-25T20:06:19.891 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:21.001 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:06:25.002 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:26.111 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:06:30.113 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:31.223 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:06:35.225 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:36.335 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:06:40.337 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:41.447 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:06:45.448 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:46.559 DEBUG:teuthology.orchestra.console:check power output: 2023-07-25T20:06:50.560 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-25T20:06:50.671 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-25T20:06:50.772 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2023-07-25T20:07:00.944 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com 2023-07-25T20:07:02.754 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 57, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/dispatcher/supervisor.py", line 227, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi161's OS to be centos 8 but found ubuntu 20.04