2023-09-19T13:07:10.656 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-19T13:07:10.657 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-19T13:07:10.847 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd136b39670>, signals=[15]) 2023-09-19T13:07:10.849 INFO:teuthology.lock.ops:Start node 'smithi100.front.sepia.ceph.com' reimaging 2023-09-19T13:07:10.849 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: reset os type and version on server 2023-09-19T13:07:10.849 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2023-09-19T13:07:10.871 INFO:teuthology.lock.ops:Start node 'smithi144.front.sepia.ceph.com' reimaging 2023-09-19T13:07:10.871 INFO:teuthology.lock.ops:Updating [smithi144.front.sepia.ceph.com]: reset os type and version on server 2023-09-19T13:07:10.871 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2023-09-19T13:07:10.873 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /host {"name": "smithi100"} 2023-09-19T13:07:10.895 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging 2023-09-19T13:07:10.896 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server 2023-09-19T13:07:10.896 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2023-09-19T13:07:10.897 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /host {"name": "smithi144"} 2023-09-19T13:07:10.920 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /host {"name": "smithi191"} 2023-09-19T13:07:10.949 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-09-19T13:07:10.960 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-09-19T13:07:10.982 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-09-19T13:07:10.994 DEBUG:teuthology.provision.fog.smithi100:Requesting image smithi_rhel_8.6 (ID 5) 2023-09-19T13:07:10.995 DEBUG:teuthology.provision.fog.smithi100:FOG request: PUT /host/205 {"imageID": 5} 2023-09-19T13:07:11.001 DEBUG:teuthology.provision.fog.smithi144:Requesting image smithi_rhel_8.6 (ID 5) 2023-09-19T13:07:11.001 DEBUG:teuthology.provision.fog.smithi144:FOG request: PUT /host/249 {"imageID": 5} 2023-09-19T13:07:11.023 DEBUG:teuthology.provision.fog.smithi191:Requesting image smithi_rhel_8.6 (ID 5) 2023-09-19T13:07:11.024 DEBUG:teuthology.provision.fog.smithi191:FOG request: PUT /host/296 {"imageID": 5} 2023-09-19T13:07:11.041 INFO:teuthology.provision.fog.smithi144:Scheduling deploy of rhel 8.6 2023-09-19T13:07:11.041 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-09-19T13:07:11.044 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of rhel 8.6 2023-09-19T13:07:11.044 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:07:11.068 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of rhel 8.6 2023-09-19T13:07:11.068 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:07:11.146 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /tasktype {"name": "deploy"} 2023-09-19T13:07:11.150 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /tasktype {"name": "deploy"} 2023-09-19T13:07:11.160 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /tasktype {"name": "deploy"} 2023-09-19T13:07:11.185 DEBUG:teuthology.provision.fog.smithi100:FOG request: POST /host/205/task {"taskTypeID": 1} 2023-09-19T13:07:11.188 DEBUG:teuthology.provision.fog.smithi144:FOG request: POST /host/249/task {"taskTypeID": 1} 2023-09-19T13:07:11.192 DEBUG:teuthology.provision.fog.smithi191:FOG request: POST /host/296/task {"taskTypeID": 1} 2023-09-19T13:07:11.250 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:07:11.255 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:07:11.258 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-09-19T13:07:11.397 INFO:teuthology.orchestra.console.smithi100:Power off 2023-09-19T13:07:11.398 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-19T13:07:11.410 INFO:teuthology.orchestra.console.smithi144:Power off 2023-09-19T13:07:11.411 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-19T13:07:11.422 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2023-09-19T13:07:11.423 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:11.433 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2023-09-19T13:07:11.434 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:11.450 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-09-19T13:07:11.462 INFO:teuthology.orchestra.console.smithi191:Power off 2023-09-19T13:07:11.464 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-19T13:07:11.474 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-09-19T13:07:11.487 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2023-09-19T13:07:11.487 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:11.512 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-09-19T13:07:15.454 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:15.475 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:15.513 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:15.563 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-09-19T13:07:15.584 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-09-19T13:07:15.623 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-09-19T13:07:19.566 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:19.585 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:19.624 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:19.675 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-09-19T13:07:19.695 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-09-19T13:07:19.733 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-09-19T13:07:19.796 INFO:teuthology.orchestra.console.smithi144:Power off completed 2023-09-19T13:07:19.897 INFO:teuthology.orchestra.console.smithi144:Power on 2023-09-19T13:07:19.898 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-19T13:07:19.921 DEBUG:teuthology.orchestra.console.smithi144:power on output: Chassis Power Control: Up/On 2023-09-19T13:07:19.922 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:19.945 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-09-19T13:07:23.678 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:23.734 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:23.788 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-09-19T13:07:23.843 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2023-09-19T13:07:23.889 INFO:teuthology.orchestra.console.smithi100:Power off completed 2023-09-19T13:07:23.945 INFO:teuthology.orchestra.console.smithi191:Power off completed 2023-09-19T13:07:23.945 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:23.991 INFO:teuthology.orchestra.console.smithi100:Power on 2023-09-19T13:07:23.992 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-19T13:07:24.012 DEBUG:teuthology.orchestra.console.smithi100:power on output: Chassis Power Control: Up/On 2023-09-19T13:07:24.014 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:24.035 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-09-19T13:07:24.046 INFO:teuthology.orchestra.console.smithi191:Power on 2023-09-19T13:07:24.046 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-19T13:07:24.054 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-09-19T13:07:24.068 DEBUG:teuthology.orchestra.console.smithi191:power on output: Chassis Power Control: Up/On 2023-09-19T13:07:24.068 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:24.090 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2023-09-19T13:07:28.038 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:28.056 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:28.091 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:28.147 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-09-19T13:07:28.164 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-09-19T13:07:28.201 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2023-09-19T13:07:28.249 INFO:teuthology.orchestra.console.smithi100:Power on completed 2023-09-19T13:07:28.266 INFO:teuthology.orchestra.console.smithi144:Power on completed 2023-09-19T13:07:28.351 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish 2023-09-19T13:07:28.351 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:07:28.367 INFO:teuthology.provision.fog.smithi144:Waiting for deploy to finish 2023-09-19T13:07:28.367 DEBUG:teuthology.provision.fog.smithi144:FOG request: GET /task/active None 2023-09-19T13:07:28.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:07:32.206 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T13:07:32.317 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-09-19T13:07:32.418 INFO:teuthology.orchestra.console.smithi191:Power on completed 2023-09-19T13:07:32.519 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish 2023-09-19T13:07:32.520 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:07:43.495 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:07:46.974 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-09-19T13:07:47.654 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:07:52.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:07:56.058 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-09-19T13:07:58.635 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:08:02.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:08:02.777 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:08:05.114 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-09-19T13:08:11.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:08:13.758 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:08:14.174 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-09-19T13:08:17.906 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:08:20.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:08:23.258 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-09-19T13:08:28.892 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:08:29.260 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:08:32.314 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-09-19T13:08:33.030 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:08:38.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:08:44.052 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:08:48.150 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:08:56.858 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-09-19T13:08:59.203 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:09:02.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:09:03.272 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:09:04.815 WARNING:teuthology.provision.fog.smithi144:[Errno None] Unable to connect to port 22 on 172.21.15.144 2023-09-19T13:09:10.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:09:10.950 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: Authentication failed. 2023-09-19T13:09:10.991 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-19T13:09:10.992 DEBUG:teuthology.orchestra.run.smithi144:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-19T13:09:10.993 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:12.000 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:13.006 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:14.012 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:14.349 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:09:15.017 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:16.023 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:17.029 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:18.036 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:18.422 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:09:19.043 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:20.049 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:21.054 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:22.060 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:23.066 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:24.072 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:25.081 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:26.086 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:27.092 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:28.097 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:29.103 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:29.524 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:09:30.112 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:31.118 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:32.124 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:33.131 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:33.600 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:09:34.138 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:35.144 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:36.150 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:37.156 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:38.162 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:39.169 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:40.175 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:41.182 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:42.188 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:43.194 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:44.201 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:44.710 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:09:45.206 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:46.212 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:47.218 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:48.224 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:48.760 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:09:49.229 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:50.236 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:51.241 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:52.247 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:53.253 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:54.259 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:55.265 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:56.271 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:57.278 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:58.286 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:59.292 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:09:59.846 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:10:00.299 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:01.305 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:02.311 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:03.317 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:03.892 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:10:04.324 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:05.330 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:06.336 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:07.343 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:08.348 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:09.354 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:10.360 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:11.366 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:12.372 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:13.379 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:14.385 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:15.015 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:10:15.392 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:16.398 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:17.404 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:18.411 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:19.055 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:10:19.418 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:20.424 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:21.430 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:22.436 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:23.443 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:24.449 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:25.455 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:26.461 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:27.469 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:28.475 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:29.481 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:30.171 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:10:30.487 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:31.492 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:32.498 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:33.505 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:34.217 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:10:34.510 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:35.517 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:36.523 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:37.529 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:38.534 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:39.540 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:40.546 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:41.552 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:42.558 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:43.563 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:44.569 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:45.313 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:10:45.575 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:46.581 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:47.587 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:48.594 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:49.357 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:10:49.599 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:50.604 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:51.610 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:52.622 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:53.626 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:54.631 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:55.636 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:56.642 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:57.648 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:58.653 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:10:59.660 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:00.451 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:11:00.666 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:01.673 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:02.678 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:03.684 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:04.479 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:11:04.690 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:05.696 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:06.701 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:07.708 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:08.713 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:09.718 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:10.723 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:10.829 ERROR:teuthology.provision.fog: on smithi144 2023-09-19T13:11:15.600 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:11:19.602 DEBUG:teuthology.provision.fog.smithi191:FOG request: GET /task/active None 2023-09-19T13:11:19.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:11:19.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-09-19T13:11:19.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:11:19.918 ERROR:teuthology.orchestra.connection:Error authenticating with smithi144.front.sepia.ceph.com: Authentication failed. 2023-09-19T13:11:19.919 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-09-19T13:11:19.921 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-09-19T13:11:30.722 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:11:38.138 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2023-09-19T13:11:44.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:11:45.830 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:11:47.194 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2023-09-19T13:11:53.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:11:56.254 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2023-09-19T13:12:00.930 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-09-19T13:12:01.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:12:02.256 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:12:05.338 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2023-09-19T13:12:11.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:12:19.354 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-09-19T13:12:25.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:12:28.410 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-09-19T13:12:34.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:12:37.466 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-09-19T13:12:43.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:12:46.526 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-09-19T13:12:52.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:12:55.582 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-09-19T13:13:01.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:13:11.343 WARNING:teuthology.provision.fog.smithi191:timed out 2023-09-19T13:13:17.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:13:17.695 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-19T13:13:17.696 DEBUG:teuthology.orchestra.run.smithi191:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-19T13:13:18.035 INFO:teuthology.provision.fog.smithi191:Node is ready 2023-09-19T13:13:18.035 DEBUG:teuthology.orchestra.run.smithi191:> hostname 2023-09-19T13:13:18.054 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com 2023-09-19T13:13:18.054 DEBUG:teuthology.orchestra.run.smithi191:> grep smithi191.front.sepia.ceph.com /etc/hosts 2023-09-19T13:13:18.113 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191 2023-09-19T13:13:18.113 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191 2023-09-19T13:13:18.241 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hosts 2023-09-19T13:13:18.271 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/172.21.15.191/172.21.15.191/g' /etc/hosts 2023-09-19T13:13:18.338 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hostname 2023-09-19T13:13:18.406 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191 2023-09-19T13:13:18.472 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2023-09-19T13:13:18.529 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux" 2023-09-19T13:13:18.530 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.6 (Ootpa)" 2023-09-19T13:13:18.530 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel" 2023-09-19T13:13:18.530 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora" 2023-09-19T13:13:18.531 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.6" 2023-09-19T13:13:18.531 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2023-09-19T13:13:18.531 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-19T13:13:18.531 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2023-09-19T13:13:18.531 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-19T13:13:18.532 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/" 2023-09-19T13:13:18.532 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-19T13:13:18.532 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-19T13:13:18.532 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-09-19T13:13:18.532 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-19T13:13:18.533 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-19T13:13:18.533 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-19T13:13:18.533 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-19T13:13:18.534 INFO:teuthology.provision.fog.smithi191:Deploy complete! 2023-09-19T13:13:18.536 DEBUG:teuthology.parallel:result is None 2023-09-19T13:13:20.026 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-09-19T13:13:26.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:13:29.082 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-09-19T13:13:35.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T13:13:37.021 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-19T13:13:37.022 DEBUG:teuthology.orchestra.run.smithi100:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-19T13:13:37.287 INFO:teuthology.provision.fog.smithi100:Node is ready 2023-09-19T13:13:37.288 DEBUG:teuthology.orchestra.run.smithi100:> hostname 2023-09-19T13:13:37.343 INFO:teuthology.orchestra.run.smithi100.stdout:smithi022 2023-09-19T13:13:37.343 DEBUG:teuthology.orchestra.run.smithi100:> grep smithi022 /etc/hosts 2023-09-19T13:13:37.399 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.22 smithi022.front.sepia.ceph.com smithi022 2023-09-19T13:13:37.399 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2023-09-19T13:13:37.543 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi022/smithi100/g' /etc/hosts 2023-09-19T13:13:37.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/172.21.15.22/172.21.15.100/g' /etc/hosts 2023-09-19T13:13:37.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi022/smithi100/g' /etc/hostname 2023-09-19T13:13:37.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2023-09-19T13:13:37.769 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2023-09-19T13:13:37.826 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Red Hat Enterprise Linux" 2023-09-19T13:13:37.826 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8.6 (Ootpa)" 2023-09-19T13:13:37.826 INFO:teuthology.orchestra.run.smithi100.stdout:ID="rhel" 2023-09-19T13:13:37.827 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="fedora" 2023-09-19T13:13:37.827 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8.6" 2023-09-19T13:13:37.827 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8" 2023-09-19T13:13:37.827 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-19T13:13:37.828 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2023-09-19T13:13:37.828 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-19T13:13:37.828 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.redhat.com/" 2023-09-19T13:13:37.828 INFO:teuthology.orchestra.run.smithi100.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-19T13:13:37.828 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-19T13:13:37.829 INFO:teuthology.orchestra.run.smithi100.stdout: 2023-09-19T13:13:37.829 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-19T13:13:37.829 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-19T13:13:37.829 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-19T13:13:37.830 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-19T13:13:37.830 INFO:teuthology.provision.fog.smithi100:Deploy complete! 2023-09-19T13:13:37.831 DEBUG:teuthology.parallel:result is None 2023-09-19T14:11:29.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-09-19T14:11:29.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-19T14:11:29.084 DEBUG:teuthology.orchestra.run.smithi144:> true 2023-09-19T14:11:29.160 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2023-09-19T14:11:29.160 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-19T14:11:29.161 DEBUG:teuthology.orchestra.run.smithi144:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-19T14:11:29.218 INFO:teuthology.provision.fog.smithi144:Node is ready 2023-09-19T14:11:29.218 DEBUG:teuthology.orchestra.run.smithi144:> hostname 2023-09-19T14:11:29.275 INFO:teuthology.orchestra.run.smithi144.stdout:smithi144 2023-09-19T14:11:29.276 DEBUG:teuthology.orchestra.run.smithi144:> grep smithi144 /etc/hosts 2023-09-19T14:11:29.332 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144 smithi144.front.sepia.ceph.com smithi144 2023-09-19T14:11:29.333 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144 smithi144 smithi144 2023-09-19T14:11:29.333 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname smithi144 2023-09-19T14:11:29.400 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/smithi144/smithi144/g' /etc/hosts 2023-09-19T14:11:29.468 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/172.21.15.144/172.21.15.144/g' /etc/hosts 2023-09-19T14:11:29.535 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i -e 's/smithi144/smithi144/g' /etc/hostname 2023-09-19T14:11:29.603 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname smithi144 2023-09-19T14:11:29.670 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2023-09-19T14:11:29.728 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2023-09-19T14:11:29.728 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="8" 2023-09-19T14:11:29.728 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2023-09-19T14:11:29.729 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2023-09-19T14:11:29.729 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="8" 2023-09-19T14:11:29.729 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el8" 2023-09-19T14:11:29.729 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-19T14:11:29.729 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2023-09-19T14:11:29.730 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-19T14:11:29.730 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2023-09-19T14:11:29.730 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-19T14:11:29.730 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-19T14:11:29.730 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-19T14:11:29.732 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Nuking machines... Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi144's OS to be rhel 8.6 but found centos 8 2023-09-19T14:11:29.733 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-19T14:11:29.901 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=7179f5c83d3d4cdbbae61ef378723b63 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi144's OS to be rhel 8.6 but found centos 8 2023-09-19T14:11:29.903 INFO:teuthology.nuke:Checking targets against current locks 2023-09-19T14:11:29.973 DEBUG:teuthology.nuke:shortname: smithi100 2023-09-19T14:11:29.974 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-19T14:11:30.000 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-09-19_13:06:06-orch:cephadm-wip-adk-testing-2023-09-18-1336-distro-default-smithi/7398099', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-19 13:07:09.815936', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-19T14:11:30.005 DEBUG:teuthology.nuke:shortname: smithi144 2023-09-19T14:11:30.005 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-19T14:11:30.027 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-09-19_13:06:06-orch:cephadm-wip-adk-testing-2023-09-18-1336-distro-default-smithi/7398161', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-19 13:56:59.888742', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJxhWBFTE/bv3kp8N2bgWPEQpx9pxq8O9CMnLa9045YD/dG4wd8xQBwOneOkfuVBdDgsZUAGP0vTaEWgRj1uw0='} 2023-09-19T14:11:30.031 DEBUG:teuthology.nuke:shortname: smithi191 2023-09-19T14:11:30.032 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-19T14:11:30.057 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2023-09-19_13:06:06-orch:cephadm-wip-adk-testing-2023-09-18-1336-distro-default-smithi/7398099', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': '', 'os_version': '', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-19 13:07:09.817959', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPBG4Z04aKlc3A8MSpb+nBBkX50jo0hiRJkFlSsCMrMcEAqNBaBJpG7WrAqn9S2XfzubCF0mPmMlSLb3GsqhgUI='} 2023-09-19T14:11:30.063 INFO:teuthology.orchestra.console.smithi100:Power off 2023-09-19T14:11:30.064 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-19T14:11:30.089 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2023-09-19T14:11:30.090 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:30.107 INFO:teuthology.orchestra.console.smithi144:Power off 2023-09-19T14:11:30.108 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-19T14:11:30.119 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-09-19T14:11:30.130 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2023-09-19T14:11:30.131 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:30.147 INFO:teuthology.orchestra.console.smithi191:Power off 2023-09-19T14:11:30.148 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-19T14:11:30.158 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-09-19T14:11:30.171 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2023-09-19T14:11:30.171 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:30.194 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-09-19T14:11:34.121 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:34.160 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:34.195 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:34.231 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-09-19T14:11:34.271 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2023-09-19T14:11:34.306 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-09-19T14:11:38.234 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:38.272 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:38.307 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:38.345 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-09-19T14:11:38.382 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2023-09-19T14:11:38.418 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-09-19T14:11:38.446 INFO:teuthology.orchestra.console.smithi100:Power off completed 2023-09-19T14:11:38.484 INFO:teuthology.orchestra.console.smithi144:Power off completed 2023-09-19T14:11:38.612 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com 2023-09-19T14:11:38.634 INFO:teuthology.lock.ops:unlocked: smithi144.front.sepia.ceph.com 2023-09-19T14:11:42.420 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-19T14:11:42.531 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2023-09-19T14:11:42.632 INFO:teuthology.orchestra.console.smithi191:Power off completed 2023-09-19T14:11:42.786 INFO:teuthology.lock.ops:unlocked: smithi191.front.sepia.ceph.com 2023-09-19T14:11:43.138 CRITICAL:teuthology:Uncaught exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology-dispatcher", line 8, in sys.exit(main()) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/scripts/dispatcher.py", line 35, in main sys.exit(teuthology.dispatcher.main(args)) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/__init__.py", line 69, in main return supervisor.main(args) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 58, in main reimage(job_config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/dispatcher/supervisor.py", line 228, in reimage reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type']) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/lock/ops.py", line 325, in reimage_machines reimaged[machine] = machines[machine] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/__init__.py", line 44, in reimage result = obj.create() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 92, in create self._verify_installed_os() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/provision/fog.py", line 348, in _verify_installed_os raise RuntimeError( RuntimeError: Expected smithi144's OS to be rhel 8.6 but found centos 8