2023-07-26T21:16:59.438 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-26T21:16:59.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:16:59.440 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:16:59.634 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4496343310>, signals=[15]) 2023-07-26T21:16:59.636 INFO:teuthology.lock.ops:Start node 'smithi062.front.sepia.ceph.com' reimaging 2023-07-26T21:16:59.636 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: reset os type and version on server 2023-07-26T21:16:59.636 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-07-26T21:16:59.665 INFO:teuthology.lock.ops:Start node 'smithi083.front.sepia.ceph.com' reimaging 2023-07-26T21:16:59.665 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: reset os type and version on server 2023-07-26T21:16:59.665 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2023-07-26T21:16:59.667 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /host {"name": "smithi062"} 2023-07-26T21:16:59.690 INFO:teuthology.lock.ops:Start node 'smithi114.front.sepia.ceph.com' reimaging 2023-07-26T21:16:59.690 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: reset os type and version on server 2023-07-26T21:16:59.691 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2023-07-26T21:16:59.692 DEBUG:teuthology.provision.fog.smithi083:FOG request: GET /host {"name": "smithi083"} 2023-07-26T21:16:59.714 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /host {"name": "smithi114"} 2023-07-26T21:16:59.740 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-26T21:16:59.768 DEBUG:teuthology.provision.fog.smithi083:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-26T21:16:59.776 DEBUG:teuthology.provision.fog.smithi062:Requesting image smithi_centos_9.stream (ID 10) 2023-07-26T21:16:59.776 DEBUG:teuthology.provision.fog.smithi062:FOG request: PUT /host/167 {"imageID": 10} 2023-07-26T21:16:59.778 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /image {"name": "smithi_centos_9.stream"} 2023-07-26T21:16:59.816 DEBUG:teuthology.provision.fog.smithi114:Requesting image smithi_centos_9.stream (ID 10) 2023-07-26T21:16:59.816 DEBUG:teuthology.provision.fog.smithi114:FOG request: PUT /host/219 {"imageID": 10} 2023-07-26T21:16:59.818 DEBUG:teuthology.provision.fog.smithi083:Requesting image smithi_centos_9.stream (ID 10) 2023-07-26T21:16:59.819 DEBUG:teuthology.provision.fog.smithi083:FOG request: PUT /host/188 {"imageID": 10} 2023-07-26T21:16:59.835 INFO:teuthology.provision.fog.smithi062:Scheduling deploy of centos 9.stream 2023-07-26T21:16:59.835 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-07-26T21:16:59.857 INFO:teuthology.provision.fog.smithi114:Scheduling deploy of centos 9.stream 2023-07-26T21:16:59.857 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-07-26T21:16:59.862 INFO:teuthology.provision.fog.smithi083:Scheduling deploy of centos 9.stream 2023-07-26T21:16:59.862 DEBUG:teuthology.provision.fog.smithi083:FOG request: GET /task/active None 2023-07-26T21:16:59.893 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /tasktype {"name": "deploy"} 2023-07-26T21:16:59.904 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /tasktype {"name": "deploy"} 2023-07-26T21:16:59.910 DEBUG:teuthology.provision.fog.smithi083:FOG request: GET /tasktype {"name": "deploy"} 2023-07-26T21:16:59.934 DEBUG:teuthology.provision.fog.smithi062:FOG request: POST /host/167/task {"taskTypeID": 1} 2023-07-26T21:16:59.937 DEBUG:teuthology.provision.fog.smithi114:FOG request: POST /host/219/task {"taskTypeID": 1} 2023-07-26T21:16:59.943 DEBUG:teuthology.provision.fog.smithi083:FOG request: POST /host/188/task {"taskTypeID": 1} 2023-07-26T21:17:00.063 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-07-26T21:17:05.070 DEBUG:teuthology.provision.fog.smithi083:FOG request: GET /task/active None 2023-07-26T21:17:05.071 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-07-26T21:17:30.192 INFO:teuthology.orchestra.console:Power off smithi083 2023-07-26T21:17:30.194 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-26T21:17:30.209 INFO:teuthology.orchestra.console:Power off smithi062 2023-07-26T21:17:30.210 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-26T21:17:30.223 INFO:teuthology.orchestra.console:Power off smithi114 2023-07-26T21:17:30.224 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-26T21:17:30.233 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-26T21:17:30.234 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:30.244 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-26T21:17:30.244 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:30.254 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-26T21:17:30.255 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:30.266 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T21:17:30.268 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T21:17:30.280 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T21:17:34.272 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:34.280 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:34.288 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:35.383 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:35.389 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:35.401 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:39.384 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:39.392 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:39.402 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:39.511 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T21:17:39.612 INFO:teuthology.orchestra.console:Power off for smithi114 completed 2023-07-26T21:17:39.713 INFO:teuthology.orchestra.console:Power on smithi114 2023-07-26T21:17:39.714 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-26T21:17:39.734 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-26T21:17:39.735 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:39.755 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T21:17:40.495 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:40.503 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:43.759 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:44.496 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:44.504 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:44.606 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T21:17:44.708 INFO:teuthology.orchestra.console:Power off for smithi062 completed 2023-07-26T21:17:44.809 INFO:teuthology.orchestra.console:Power on smithi062 2023-07-26T21:17:44.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-26T21:17:44.828 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-26T21:17:44.829 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:44.853 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T21:17:44.870 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:45.615 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:48.854 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:48.871 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:49.617 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:49.964 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:49.981 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:50.727 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:53.966 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:53.982 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:54.078 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T21:17:54.179 INFO:teuthology.orchestra.console:Power on for smithi062 completed 2023-07-26T21:17:54.280 INFO:teuthology.provision.fog.smithi062:Waiting for deploy to finish 2023-07-26T21:17:54.281 DEBUG:teuthology.provision.fog.smithi062:FOG request: GET /task/active None 2023-07-26T21:17:56.607 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:17:56.616 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:56.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:17:57.718 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:17:59.728 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-26T21:18:00.618 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:01.720 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:01.728 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:02.830 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:05.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:18:10.737 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:10.746 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:11.848 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:11.855 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:13.808 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-26T21:18:15.849 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:15.857 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:16.959 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:16.967 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:19.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:18:24.817 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:24.825 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:25.928 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:25.936 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:27.888 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-26T21:18:29.932 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:29.939 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:31.041 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:31.047 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:33.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:18:35.043 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:35.050 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:36.152 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:36.159 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:36.944 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-26T21:18:40.153 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi114.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:40.163 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:40.265 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T21:18:40.366 INFO:teuthology.orchestra.console:Power on for smithi114 completed 2023-07-26T21:18:40.467 INFO:teuthology.provision.fog.smithi114:Waiting for deploy to finish 2023-07-26T21:18:40.468 DEBUG:teuthology.provision.fog.smithi114:FOG request: GET /task/active None 2023-07-26T21:18:45.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:18:50.480 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:50.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:18:53.552 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-26T21:18:53.616 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-07-26T21:18:54.482 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:18:55.594 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:18:59.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:19:04.561 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:19:04.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:19:09.587 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-26T21:19:09.688 INFO:teuthology.orchestra.console:Power off for smithi083 completed 2023-07-26T21:19:09.789 INFO:teuthology.orchestra.console:Power on smithi083 2023-07-26T21:19:09.789 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-26T21:19:12.656 WARNING:teuthology.provision.fog.smithi114:[Errno None] Unable to connect to port 22 on 172.21.15.114 2023-07-26T21:19:18.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:19:23.665 WARNING:teuthology.provision.fog.smithi062:[Errno None] Unable to connect to port 22 on 172.21.15.62 2023-07-26T21:19:23.668 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-26T21:19:23.668 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:19:23.789 ERROR:teuthology.orchestra.connection:Error authenticating with smithi114.front.sepia.ceph.com: Authentication failed. 2023-07-26T21:19:23.791 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-26T21:19:23.792 DEBUG:teuthology.orchestra.run.smithi114:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-26T21:19:23.792 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:24.677 DEBUG:teuthology.orchestra.console:check power output: 2023-07-26T21:19:24.798 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:25.804 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:26.809 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:27.815 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:28.678 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-26T21:19:28.790 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-26T21:19:28.823 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:28.892 INFO:teuthology.orchestra.console:Power on for smithi083 completed 2023-07-26T21:19:28.993 INFO:teuthology.provision.fog.smithi083:Waiting for deploy to finish 2023-07-26T21:19:28.994 DEBUG:teuthology.provision.fog.smithi083:FOG request: GET /task/active None 2023-07-26T21:19:29.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:19:29.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:19:29.747 ERROR:teuthology.orchestra.connection:Error authenticating with smithi062.front.sepia.ceph.com: Authentication failed. 2023-07-26T21:19:29.747 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-26T21:19:29.748 DEBUG:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-26T21:19:29.749 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:29.830 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:30.756 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:30.837 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:31.765 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:31.847 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:32.144 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2023-07-26T21:19:32.774 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:32.855 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:33.784 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:33.863 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:34.793 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:34.872 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:35.802 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:35.881 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:36.811 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:36.890 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:37.820 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:37.899 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:38.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:19:38.828 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:38.908 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:39.836 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:39.916 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:40.845 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:40.926 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:41.200 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2023-07-26T21:19:41.853 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:41.933 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:42.862 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:42.943 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:43.872 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:43.952 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:44.881 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:44.961 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:45.889 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:45.969 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:46.898 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:46.978 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:47.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:19:52.205 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:52.206 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:53.210 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:53.210 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:54.216 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:54.217 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:55.222 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:55.223 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:55.280 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2023-07-26T21:19:56.228 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:56.228 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:57.234 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:57.235 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:58.241 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:58.242 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:59.250 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:19:59.250 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:00.257 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:00.257 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:01.264 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:01.265 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:01.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:20:02.271 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:02.272 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:03.278 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:03.279 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:04.285 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:04.285 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:04.336 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2023-07-26T21:20:05.292 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:05.292 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:06.299 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:06.300 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:07.307 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:07.308 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:08.313 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:08.313 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:09.319 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:09.319 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:10.327 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:10.327 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:10.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:20:15.341 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:15.342 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:16.347 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:17.353 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:18.360 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:20:18.416 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2023-07-26T21:20:24.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:20:27.472 WARNING:teuthology.provision.fog.smithi083:[Errno None] Unable to connect to port 22 on 172.21.15.83 2023-07-26T21:20:33.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:21:10.070 ERROR:teuthology.orchestra.connection:Error authenticating with smithi083.front.sepia.ceph.com: Authentication failed. 2023-07-26T21:21:10.070 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-26T21:21:10.071 DEBUG:teuthology.orchestra.run.smithi083:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-26T21:21:10.072 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:11.081 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:12.089 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:13.098 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:14.107 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:15.115 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:16.124 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:17.132 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:18.141 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:19.151 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:20.159 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:21.167 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:22.176 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:23.185 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:24.194 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:25.203 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:26.212 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:27.221 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:28.231 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:29.240 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:30.249 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:31.259 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:32.269 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:33.279 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:34.289 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:35.298 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:36.306 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:37.316 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:38.325 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:39.333 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:40.341 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:41.350 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:42.360 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:43.367 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:44.375 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:45.383 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:46.392 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:47.402 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:48.410 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:49.420 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:50.427 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:51.436 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:52.445 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:53.454 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:54.463 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:21:55.472 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-07-26T21:22:02.992 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-26T21:22:02.993 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi114 2023-07-26T21:22:07.088 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-26T21:22:07.089 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi062 2023-07-26T21:22:11.995 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-07-26T21:22:11.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:22:12.303 DEBUG:teuthology.orchestra.run.smithi114:> true 2023-07-26T21:22:12.601 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-07-26T21:22:12.601 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-26T21:22:12.602 DEBUG:teuthology.orchestra.run.smithi114:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-26T21:22:16.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-26T21:22:16.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:22:16.344 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-07-26T21:22:16.695 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-26T21:22:16.696 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-26T21:22:16.696 DEBUG:teuthology.orchestra.run.smithi062:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-26T21:22:17.616 INFO:teuthology.provision.fog.smithi114:Node is ready 2023-07-26T21:22:17.617 DEBUG:teuthology.orchestra.run.smithi114:> hostname 2023-07-26T21:22:17.634 INFO:teuthology.orchestra.run.smithi114.stdout:smithi197 2023-07-26T21:22:17.635 DEBUG:teuthology.orchestra.run.smithi114:> grep smithi197 /etc/hosts 2023-07-26T21:22:17.692 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.197 smithi197.front.sepia.ceph.com smithi197 2023-07-26T21:22:17.692 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114 2023-07-26T21:22:17.831 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi197/smithi114/g' /etc/hosts 2023-07-26T21:22:17.863 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/172.21.15.197/172.21.15.114/g' /etc/hosts 2023-07-26T21:22:17.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i -e 's/smithi197/smithi114/g' /etc/hostname 2023-07-26T21:22:18.004 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname smithi114 2023-07-26T21:22:18.072 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2023-07-26T21:22:18.128 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2023-07-26T21:22:18.128 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2023-07-26T21:22:18.128 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2023-07-26T21:22:18.128 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2023-07-26T21:22:18.128 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2023-07-26T21:22:18.129 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2023-07-26T21:22:18.129 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-26T21:22:18.129 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2023-07-26T21:22:18.129 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2023-07-26T21:22:18.129 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-26T21:22:18.129 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2023-07-26T21:22:18.130 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-26T21:22:18.130 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-26T21:22:18.130 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-26T21:22:18.131 INFO:teuthology.provision.fog.smithi114:Deploy complete! 2023-07-26T21:22:18.132 DEBUG:teuthology.parallel:result is None 2023-07-26T21:22:21.712 INFO:teuthology.provision.fog.smithi062:Node is ready 2023-07-26T21:22:21.712 DEBUG:teuthology.orchestra.run.smithi062:> hostname 2023-07-26T21:22:21.730 INFO:teuthology.orchestra.run.smithi062.stdout:smithi062.front.sepia.ceph.com 2023-07-26T21:22:21.730 DEBUG:teuthology.orchestra.run.smithi062:> grep smithi062.front.sepia.ceph.com /etc/hosts 2023-07-26T21:22:21.789 INFO:teuthology.orchestra.run.smithi062.stdout:172.21.15.62 smithi062.front.sepia.ceph.com smithi062 2023-07-26T21:22:21.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2023-07-26T21:22:21.927 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hosts 2023-07-26T21:22:21.963 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/172.21.15.62/172.21.15.62/g' /etc/hosts 2023-07-26T21:22:22.038 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i -e 's/smithi062.front.sepia.ceph.com/smithi062/g' /etc/hostname 2023-07-26T21:22:22.111 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname smithi062 2023-07-26T21:22:22.184 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-07-26T21:22:22.241 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2023-07-26T21:22:22.241 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2023-07-26T21:22:22.241 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2023-07-26T21:22:22.241 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2023-07-26T21:22:22.242 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2023-07-26T21:22:22.242 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2023-07-26T21:22:22.242 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-26T21:22:22.242 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-07-26T21:22:22.242 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2023-07-26T21:22:22.242 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-26T21:22:22.242 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2023-07-26T21:22:22.243 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-26T21:22:22.243 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-26T21:22:22.243 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-26T21:22:22.243 INFO:teuthology.provision.fog.smithi062:Deploy complete! 2023-07-26T21:22:22.244 DEBUG:teuthology.parallel:result is None 2023-07-26T21:23:43.344 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-07-26T21:23:43.345 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi083 2023-07-26T21:23:52.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-07-26T21:23:52.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:23:52.604 DEBUG:teuthology.orchestra.run.smithi083:> true 2023-07-26T21:23:52.942 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-07-26T21:23:52.942 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-26T21:23:52.943 DEBUG:teuthology.orchestra.run.smithi083:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-26T21:23:57.959 INFO:teuthology.provision.fog.smithi083:Node is ready 2023-07-26T21:23:57.959 DEBUG:teuthology.orchestra.run.smithi083:> hostname 2023-07-26T21:23:57.978 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083.front.sepia.ceph.com 2023-07-26T21:23:57.979 DEBUG:teuthology.orchestra.run.smithi083:> grep smithi083.front.sepia.ceph.com /etc/hosts 2023-07-26T21:23:58.035 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83 smithi083.front.sepia.ceph.com smithi083 2023-07-26T21:23:58.035 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname smithi083 2023-07-26T21:23:58.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i -e 's/smithi083.front.sepia.ceph.com/smithi083/g' /etc/hosts 2023-07-26T21:23:58.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i -e 's/172.21.15.83/172.21.15.83/g' /etc/hosts 2023-07-26T21:23:58.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i -e 's/smithi083.front.sepia.ceph.com/smithi083/g' /etc/hostname 2023-07-26T21:23:58.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname smithi083 2023-07-26T21:23:58.390 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2023-07-26T21:23:58.445 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2023-07-26T21:23:58.446 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2023-07-26T21:23:58.446 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2023-07-26T21:23:58.446 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2023-07-26T21:23:58.446 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2023-07-26T21:23:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2023-07-26T21:23:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-26T21:23:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2023-07-26T21:23:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2023-07-26T21:23:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-26T21:23:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2023-07-26T21:23:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-26T21:23:58.447 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-26T21:23:58.448 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-26T21:23:58.448 INFO:teuthology.provision.fog.smithi083:Deploy complete! 2023-07-26T21:23:58.448 DEBUG:teuthology.parallel:result is None 2023-07-26T21:23:58.648 INFO:teuthology.lock.ops:Checking smithi062.front.sepia.ceph.com 2023-07-26T21:23:58.649 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-26T21:23:58.688 INFO:teuthology.lock.ops:Checking smithi083.front.sepia.ceph.com 2023-07-26T21:23:58.689 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-26T21:23:58.710 INFO:teuthology.lock.ops:Checking smithi114.front.sepia.ceph.com 2023-07-26T21:23:58.710 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-26T21:23:58.728 INFO:teuthology.lock.ops:Updating [smithi062.front.sepia.ceph.com]: set os type and version on server 2023-07-26T21:23:58.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-26T21:23:58.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:23:58.828 DEBUG:teuthology.orchestra.run.smithi062:> true 2023-07-26T21:23:59.015 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2023-07-26T21:23:59.015 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2023-07-26T21:23:59.070 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2023-07-26T21:23:59.070 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2023-07-26T21:23:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2023-07-26T21:23:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2023-07-26T21:23:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2023-07-26T21:23:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2023-07-26T21:23:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2023-07-26T21:23:59.125 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2023-07-26T21:23:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-26T21:23:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2023-07-26T21:23:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2023-07-26T21:23:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-26T21:23:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2023-07-26T21:23:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-26T21:23:59.126 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-26T21:23:59.127 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-26T21:23:59.127 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2023-07-26T21:23:59.152 INFO:teuthology.lock.ops:Updating [smithi083.front.sepia.ceph.com]: set os type and version on server 2023-07-26T21:23:59.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-07-26T21:23:59.153 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:23:59.235 DEBUG:teuthology.orchestra.run.smithi083:> true 2023-07-26T21:23:59.304 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2023-07-26T21:23:59.305 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2023-07-26T21:23:59.360 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2023-07-26T21:23:59.361 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2023-07-26T21:23:59.416 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2023-07-26T21:23:59.417 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2023-07-26T21:23:59.417 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2023-07-26T21:23:59.417 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2023-07-26T21:23:59.417 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2023-07-26T21:23:59.418 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2023-07-26T21:23:59.418 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-26T21:23:59.418 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2023-07-26T21:23:59.418 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2023-07-26T21:23:59.418 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-26T21:23:59.418 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2023-07-26T21:23:59.418 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-26T21:23:59.419 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-26T21:23:59.419 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-26T21:23:59.419 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2023-07-26T21:23:59.444 INFO:teuthology.lock.ops:Updating [smithi114.front.sepia.ceph.com]: set os type and version on server 2023-07-26T21:23:59.444 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-07-26T21:23:59.444 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-26T21:23:59.528 DEBUG:teuthology.orchestra.run.smithi114:> true 2023-07-26T21:23:59.698 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2023-07-26T21:23:59.699 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2023-07-26T21:23:59.753 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2023-07-26T21:23:59.754 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2023-07-26T21:23:59.807 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2023-07-26T21:23:59.807 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2023-07-26T21:23:59.807 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2023-07-26T21:23:59.807 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2023-07-26T21:23:59.807 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2023-07-26T21:23:59.807 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2023-07-26T21:23:59.808 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2023-07-26T21:23:59.808 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2023-07-26T21:23:59.808 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2023-07-26T21:23:59.808 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-07-26T21:23:59.808 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2023-07-26T21:23:59.808 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-26T21:23:59.808 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-07-26T21:23:59.809 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-07-26T21:23:59.809 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2023-07-26T21:24:01.760 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:24:01.761 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:24:01.865 INFO:teuthology.dispatcher.supervisor:Running job 7353124 2023-07-26T21:24:01.865 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-26_14:34:38-fs-reef-release-distro-default-smithi/7353124 --name yuriw-2023-07-26_14:34:38-fs-reef-release-distro-default-smithi --description fs/upgrade/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} -- /home/teuthworker/archive/yuriw-2023-07-26_14:34:38-fs-reef-release-distro-default-smithi/7353124/orig.config.yaml 2023-07-26T21:24:01.869 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-26_14:34:38-fs-reef-release-distro-default-smithi/7353124 2023-07-26T21:24:01.870 INFO:teuthology.dispatcher.supervisor:Job PID: 24930 2023-07-26T21:24:01.870 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-26T21:26:01.971 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:26:01.972 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:28:07.148 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:28:07.149 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:30:12.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:30:12.330 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:32:17.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:32:17.514 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:34:22.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:34:22.689 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:36:27.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:36:27.866 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:38:33.046 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-26T21:38:33.047 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-26T21:38:34.854 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120