2023-08-15T23:37:50.607 INFO:root:teuthology version: 0.0.1.dev183+g5435569 2023-08-15T23:37:50.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:38:15.763 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f00c9232f70>, signals=[15]) 2023-08-15T23:38:15.765 INFO:teuthology.lock.ops:Start node 'smithi100.front.sepia.ceph.com' reimaging 2023-08-15T23:38:15.766 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: reset os type and version on server 2023-08-15T23:38:15.766 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2023-08-15T23:38:20.836 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging 2023-08-15T23:38:20.837 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server 2023-08-15T23:38:20.837 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-08-15T23:38:20.838 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /host {"name": "smithi100"} 2023-08-15T23:38:25.867 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /host {"name": "smithi154"} 2023-08-15T23:38:35.918 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-15T23:38:40.582 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /image {"name": "smithi_centos_8.stream"} 2023-08-15T23:38:40.625 DEBUG:teuthology.provision.fog.smithi154:Requesting image smithi_centos_8.stream (ID 3) 2023-08-15T23:38:40.625 DEBUG:teuthology.provision.fog.smithi154:FOG request: PUT /host/259 {"imageID": 3} 2023-08-15T23:38:40.628 DEBUG:teuthology.provision.fog.smithi100:Requesting image smithi_centos_8.stream (ID 3) 2023-08-15T23:38:40.628 DEBUG:teuthology.provision.fog.smithi100:FOG request: PUT /host/205 {"imageID": 3} 2023-08-15T23:38:40.682 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of centos 8.stream 2023-08-15T23:38:40.682 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None 2023-08-15T23:38:40.688 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of centos 8.stream 2023-08-15T23:38:40.688 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:38:40.743 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /tasktype {"name": "deploy"} 2023-08-15T23:38:40.752 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /tasktype {"name": "deploy"} 2023-08-15T23:38:40.778 DEBUG:teuthology.provision.fog.smithi154:FOG request: POST /host/259/task {"taskTypeID": 1} 2023-08-15T23:38:40.785 DEBUG:teuthology.provision.fog.smithi100:FOG request: POST /host/205/task {"taskTypeID": 1} 2023-08-15T23:38:40.840 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None 2023-08-15T23:38:40.850 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:38:45.600 INFO:teuthology.orchestra.console.smithi154:Power off 2023-08-15T23:38:45.601 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T23:38:45.619 INFO:teuthology.orchestra.console.smithi100:Power off 2023-08-15T23:38:45.620 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-15T23:38:45.630 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2023-08-15T23:38:45.631 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:45.644 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2023-08-15T23:38:45.645 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:45.655 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-08-15T23:38:45.666 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-08-15T23:38:45.756 INFO:teuthology.orchestra.console.smithi154:Power off completed 2023-08-15T23:38:45.767 INFO:teuthology.orchestra.console.smithi100:Power off completed 2023-08-15T23:38:45.858 INFO:teuthology.orchestra.console.smithi154:Power on 2023-08-15T23:38:45.858 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-15T23:38:45.868 INFO:teuthology.orchestra.console.smithi100:Power on 2023-08-15T23:38:45.869 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-08-15T23:38:45.886 DEBUG:teuthology.orchestra.console.smithi154:power on output: Chassis Power Control: Up/On 2023-08-15T23:38:45.887 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:45.899 DEBUG:teuthology.orchestra.console.smithi100:power on output: Chassis Power Control: Up/On 2023-08-15T23:38:45.900 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:45.910 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-08-15T23:38:45.921 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-08-15T23:38:49.912 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:49.922 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:50.023 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-08-15T23:38:50.033 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-08-15T23:38:54.024 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:54.035 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:55.134 DEBUG:teuthology.orchestra.console.smithi154:check power output: 2023-08-15T23:38:55.144 DEBUG:teuthology.orchestra.console.smithi100:check power output: 2023-08-15T23:38:59.135 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:59.145 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-15T23:38:59.254 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2023-08-15T23:38:59.355 INFO:teuthology.orchestra.console.smithi100:Power on completed 2023-08-15T23:38:59.456 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish 2023-08-15T23:38:59.457 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:39:04.465 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-08-15T23:39:04.566 INFO:teuthology.orchestra.console.smithi154:Power on completed 2023-08-15T23:39:04.667 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish 2023-08-15T23:39:04.667 DEBUG:teuthology.provision.fog.smithi154:FOG request: GET /task/active None 2023-08-15T23:39:04.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:39:07.792 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-08-15T23:39:13.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:39:16.848 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-08-15T23:39:19.538 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:39:22.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:39:25.904 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-08-15T23:39:31.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:39:34.960 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-08-15T23:39:36.064 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:39:41.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:39:49.136 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-08-15T23:39:55.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:40:01.136 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:40:06.140 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-08-15T23:40:12.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:40:21.208 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:40:35.572 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-08-15T23:40:40.144 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:40:45.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:40:45.264 ERROR:teuthology.orchestra.connection:Error authenticating with smithi154.front.sepia.ceph.com: Authentication failed. 2023-08-15T23:40:45.266 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-15T23:40:45.266 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-15T23:40:45.267 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:46.272 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:47.277 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:48.282 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:49.287 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:50.293 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:51.298 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:52.305 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:53.310 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:54.316 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:55.321 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:56.326 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:57.332 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:58.337 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:40:59.342 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:00.228 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:41:05.237 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:06.242 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:07.247 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:08.252 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:09.257 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:10.262 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:11.268 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:12.273 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:13.279 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:14.284 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:15.289 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:16.295 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:17.300 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:18.306 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:19.312 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:20.306 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:41:25.313 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:26.317 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:27.321 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:28.325 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:29.330 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:30.335 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:31.340 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:32.346 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:33.351 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:34.356 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:35.362 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:36.368 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:37.373 WARNING:paramiko.transport:Oops, unhandled type 3 ('unimplemented') 2023-08-15T23:41:40.406 DEBUG:teuthology.provision.fog.smithi100:FOG request: GET /task/active None 2023-08-15T23:41:40.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:41:48.564 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-08-15T23:41:54.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:42:02.640 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-08-15T23:42:08.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:42:16.724 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-08-15T23:42:22.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:42:30.800 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-08-15T23:42:36.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:43:00.212 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2023-08-15T23:43:06.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:43:11.778 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-15T23:43:11.779 DEBUG:teuthology.orchestra.run.smithi100:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-15T23:43:37.271 INFO:teuthology.provision.fog.smithi100:Node is ready 2023-08-15T23:43:37.271 DEBUG:teuthology.orchestra.run.smithi100:> hostname 2023-08-15T23:43:37.295 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com 2023-08-15T23:43:37.296 DEBUG:teuthology.orchestra.run.smithi100:> grep smithi100.front.sepia.ceph.com /etc/hosts 2023-08-15T23:43:37.356 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100 2023-08-15T23:43:37.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2023-08-15T23:43:37.473 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi100.front.sepia.ceph.com/smithi100/g' /etc/hosts 2023-08-15T23:43:37.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/172.21.15.100/172.21.15.100/g' /etc/hosts 2023-08-15T23:43:37.572 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi100.front.sepia.ceph.com/smithi100/g' /etc/hostname 2023-08-15T23:43:37.641 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2023-08-15T23:43:37.709 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2023-08-15T23:43:37.767 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2023-08-15T23:43:37.767 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8" 2023-08-15T23:43:37.767 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2023-08-15T23:43:37.767 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2023-08-15T23:43:37.768 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8" 2023-08-15T23:43:37.768 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8" 2023-08-15T23:43:37.768 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T23:43:37.768 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2023-08-15T23:43:37.768 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T23:43:37.768 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2023-08-15T23:43:37.768 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T23:43:37.769 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T23:43:37.769 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T23:43:37.769 INFO:teuthology.provision.fog.smithi100:Deploy complete! 2023-08-15T23:43:37.770 DEBUG:teuthology.parallel:result is None 2023-08-15T23:45:12.304 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-08-15T23:45:12.306 ERROR:teuthology.provision.fog:[Errno 104] Connection reset by peer on smithi154 2023-08-15T23:45:21.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-08-15T23:45:21.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:45:26.531 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-08-15T23:45:26.816 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-08-15T23:45:26.817 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-08-15T23:45:26.817 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-08-15T23:45:26.838 INFO:teuthology.provision.fog.smithi154:Node is ready 2023-08-15T23:45:26.838 DEBUG:teuthology.orchestra.run.smithi154:> hostname 2023-08-15T23:45:26.898 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com 2023-08-15T23:45:26.898 DEBUG:teuthology.orchestra.run.smithi154:> grep smithi154.front.sepia.ceph.com /etc/hosts 2023-08-15T23:45:26.958 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154 2023-08-15T23:45:26.959 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154 2023-08-15T23:45:27.071 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hosts 2023-08-15T23:45:27.103 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/172.21.15.154/172.21.15.154/g' /etc/hosts 2023-08-15T23:45:27.173 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hostname 2023-08-15T23:45:27.243 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154 2023-08-15T23:45:27.311 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-08-15T23:45:27.370 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2023-08-15T23:45:27.370 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8" 2023-08-15T23:45:27.370 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2023-08-15T23:45:27.370 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2023-08-15T23:45:27.371 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8" 2023-08-15T23:45:27.371 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8" 2023-08-15T23:45:27.371 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T23:45:27.371 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2023-08-15T23:45:27.371 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T23:45:27.371 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2023-08-15T23:45:27.371 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T23:45:27.372 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T23:45:27.372 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T23:45:27.372 INFO:teuthology.provision.fog.smithi154:Deploy complete! 2023-08-15T23:45:27.373 DEBUG:teuthology.parallel:result is None 2023-08-15T23:45:47.024 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com 2023-08-15T23:45:47.025 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-15T23:45:52.051 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com 2023-08-15T23:45:52.051 INFO:teuthology.lock.ops:New key found. Updating... 2023-08-15T23:45:57.087 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server 2023-08-15T23:45:57.087 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-08-15T23:45:57.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:45:57.157 DEBUG:teuthology.orchestra.run.smithi100:> true 2023-08-15T23:45:57.292 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2023-08-15T23:45:57.293 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2023-08-15T23:45:57.351 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2023-08-15T23:45:57.352 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2023-08-15T23:45:57.410 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2023-08-15T23:45:57.410 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8" 2023-08-15T23:45:57.411 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2023-08-15T23:45:57.411 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2023-08-15T23:45:57.411 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8" 2023-08-15T23:45:57.411 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8" 2023-08-15T23:45:57.412 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T23:45:57.412 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2023-08-15T23:45:57.412 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T23:45:57.412 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2023-08-15T23:45:57.412 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T23:45:57.412 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T23:45:57.413 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T23:45:57.413 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2023-08-15T23:45:59.275 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server 2023-08-15T23:45:59.276 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-08-15T23:45:59.277 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-15T23:45:59.350 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-08-15T23:45:59.441 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-08-15T23:45:59.441 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2023-08-15T23:45:59.500 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2023-08-15T23:45:59.501 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-08-15T23:45:59.559 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2023-08-15T23:45:59.560 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8" 2023-08-15T23:45:59.560 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2023-08-15T23:45:59.560 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2023-08-15T23:45:59.560 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8" 2023-08-15T23:45:59.561 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8" 2023-08-15T23:45:59.561 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 8" 2023-08-15T23:45:59.561 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2023-08-15T23:45:59.562 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-08-15T23:45:59.562 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2023-08-15T23:45:59.562 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-15T23:45:59.562 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-15T23:45:59.563 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-15T23:45:59.563 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-08-15T23:46:04.591 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:46:09.675 INFO:teuthology.dispatcher.supervisor:Running job 7369016 2023-08-15T23:46:09.675 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54355694caed5e99f46f0c9317b1e9bcf8940415/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-08-15_17:00:18-rbd-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7369016 --name yuriw-2023-08-15_17:00:18-rbd-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi --description rbd/thrash/{base/install clusters/{fixed-2 openstack} msgr-failures/few objectstore/bluestore-comp-zlib supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} -- /home/teuthworker/archive/yuriw-2023-08-15_17:00:18-rbd-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7369016/orig.config.yaml 2023-08-15T23:46:09.680 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-08-15_17:00:18-rbd-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7369016 2023-08-15T23:46:09.680 INFO:teuthology.dispatcher.supervisor:Job PID: 5646 2023-08-15T23:46:09.681 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-08-15T23:48:09.780 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:50:14.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:52:20.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:54:24.957 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:56:30.129 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-15T23:58:35.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-16T00:00:40.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-16T00:02:45.672 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-16T00:04:46.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-16T00:06:46.234 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-16T00:08:51.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-16T00:10:56.565 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-16T00:13:01.749 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-16T00:13:06.814 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-08-16T00:13:11.288 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-08-16T00:13:11.288 INFO:teuthology.nuke:Checking targets against current locks 2023-08-16T00:13:11.324 DEBUG:teuthology.nuke:shortname: smithi100 2023-08-16T00:13:11.325 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-16T00:13:11.345 DEBUG:teuthology.nuke:shortname: smithi154 2023-08-16T00:13:11.345 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-16T00:13:11.348 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-15_17:00:18-rbd-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7369016', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-15 23:37:49.805865', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJiQVI5DZB6vsDXx73JFjEUynNFiPVxOl6jevRQ0+FRKFNV5L48x1CUJ7u3YeGNhTKCtQxAWDkiWbWW46sw1Rp4mp8SrzyPewVRVrvxlLdfJCNx13WvmNna99CocGRVcTyLHcQMler8sp5+ZXrTaAuLexLpvT5RfXdgwzLVuB9dqamrfKNu6N0BhSgs5/3fAEFz7q99r4XHFuaGlSN6u9YdQiCukqXyXQe21AZCa3Y0+EthmWBDM07oZ+MC4AxFz2G22jLdLIM1QDRVo667mQw4Hz3ZtvyAF1ZiKcJJ6yk7G540u6ck0C1IB2dW7Yg7rfXszawifClNTodTPaKuO2EDrxK1qIKjalwJONH6g9UihetdCuq83Iz51a4jaEnEfv21ezk3cnN7szO4lsjc+jvfPCO9Lp18B+oPpCM4vTKpGEyivRlLN5L0aGPbkU1rVryI0odTSMOfc7XdWtRTWxaYeVAWEnhJAT02QJmDsWPG3/7PceAZSkhnMQIuchyOIM='} 2023-08-16T00:13:11.367 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-08-15_17:00:18-rbd-wip-yuri7-testing-2023-08-15-0741-pacific-distro-default-smithi/7369016', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-15 23:37:49.807365', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNbUfg6Rr1FPWT1jfLNs4WCkLOnCRGAG4t/ZZRT8DP4StEKXET4VU1NDMS7ufZRlEykDBVEqnapBsUallDJ3jJ0='} 2023-08-16T00:13:11.394 INFO:teuthology.orchestra.console.smithi100:Power off 2023-08-16T00:13:11.395 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-16T00:13:11.419 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2023-08-16T00:13:11.420 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-16T00:13:11.435 INFO:teuthology.orchestra.console.smithi154:Power off 2023-08-16T00:13:11.436 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-16T00:13:11.447 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2023-08-16T00:13:11.460 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2023-08-16T00:13:11.460 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-16T00:13:11.481 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-08-16T00:13:11.548 INFO:teuthology.orchestra.console.smithi100:Power off completed 2023-08-16T00:13:11.584 INFO:teuthology.orchestra.console.smithi154:Power off completed 2023-08-16T00:13:31.721 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com 2023-08-16T00:13:31.729 INFO:teuthology.lock.ops:unlocked: smithi100.front.sepia.ceph.com