2023-07-27T14:07:51.617 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-27T14:07:51.619 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:07:51.619 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:07:51.785 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f771057c310>, signals=[15]) 2023-07-27T14:07:51.787 INFO:teuthology.lock.ops:Start node 'smithi120.front.sepia.ceph.com' reimaging 2023-07-27T14:07:51.788 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: reset os type and version on server 2023-07-27T14:07:51.789 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2023-07-27T14:07:51.819 INFO:teuthology.lock.ops:Start node 'smithi132.front.sepia.ceph.com' reimaging 2023-07-27T14:07:51.820 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: reset os type and version on server 2023-07-27T14:07:51.821 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-07-27T14:07:51.822 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /host {"name": "smithi120"} 2023-07-27T14:07:51.851 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /host {"name": "smithi132"} 2023-07-27T14:07:56.924 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-27T14:07:56.927 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-07-27T14:07:56.973 DEBUG:teuthology.provision.fog.smithi132:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-27T14:07:56.974 DEBUG:teuthology.provision.fog.smithi132:FOG request: PUT /host/237 {"imageID": 7} 2023-07-27T14:07:56.979 DEBUG:teuthology.provision.fog.smithi120:Requesting image smithi_rhel_8.4 (ID 7) 2023-07-27T14:07:56.980 DEBUG:teuthology.provision.fog.smithi120:FOG request: PUT /host/225 {"imageID": 7} 2023-07-27T14:07:57.024 INFO:teuthology.provision.fog.smithi132:Scheduling deploy of rhel 8.4 2023-07-27T14:07:57.024 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:07:57.029 INFO:teuthology.provision.fog.smithi120:Scheduling deploy of rhel 8.4 2023-07-27T14:07:57.029 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:07:57.093 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /tasktype {"name": "deploy"} 2023-07-27T14:07:57.101 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /tasktype {"name": "deploy"} 2023-07-27T14:07:57.131 DEBUG:teuthology.provision.fog.smithi132:FOG request: POST /host/237/task {"taskTypeID": 1} 2023-07-27T14:07:57.134 DEBUG:teuthology.provision.fog.smithi120:FOG request: POST /host/225/task {"taskTypeID": 1} 2023-07-27T14:07:57.193 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:07:57.195 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:07:57.301 INFO:teuthology.orchestra.console:Power off smithi132 2023-07-27T14:07:57.302 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T14:07:57.320 INFO:teuthology.orchestra.console:Power off smithi120 2023-07-27T14:07:57.321 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T14:07:57.331 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T14:07:57.331 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:07:57.342 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T14:07:57.343 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:07:57.356 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T14:07:57.370 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T14:08:01.360 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:01.371 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:02.472 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:02.483 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:06.473 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:06.484 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:07.584 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:07.593 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:11.589 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:11.598 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:12.702 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:12.709 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:16.704 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:16.712 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:17.815 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:17.822 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:21.817 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:21.826 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:22.928 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:22.935 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:26.933 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:26.941 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:28.044 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:28.052 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:32.046 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:32.055 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:33.158 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:33.165 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:37.159 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:37.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:37.279 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T14:08:37.380 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2023-07-27T14:08:37.480 INFO:teuthology.orchestra.console:Power on smithi120 2023-07-27T14:08:37.481 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-27T14:08:38.270 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:42.272 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:42.383 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T14:08:42.484 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2023-07-27T14:08:42.507 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-27T14:08:42.507 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:42.585 INFO:teuthology.orchestra.console:Power on smithi132 2023-07-27T14:08:42.585 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-07-27T14:08:42.608 DEBUG:teuthology.orchestra.console:power on output: Chassis Power Control: Up/On 2023-07-27T14:08:42.608 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:42.636 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T14:08:43.516 DEBUG:teuthology.orchestra.console:check power output: 2023-07-27T14:08:46.640 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:46.751 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T14:08:47.518 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:47.629 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T14:08:47.730 INFO:teuthology.orchestra.console:Power on for smithi120 completed 2023-07-27T14:08:47.831 INFO:teuthology.provision.fog.smithi120:Waiting for deploy to finish 2023-07-27T14:08:47.832 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:08:52.479 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:08:52.589 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is on 2023-07-27T14:08:52.690 INFO:teuthology.orchestra.console:Power on for smithi132 completed 2023-07-27T14:08:52.791 INFO:teuthology.provision.fog.smithi132:Waiting for deploy to finish 2023-07-27T14:08:52.792 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:09:07.584 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:09:12.575 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:09:37.656 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:09:37.658 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:09:57.748 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:09:57.750 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:10:12.828 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:10:12.829 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:10:32.916 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:10:32.917 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:10:58.000 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:10:58.002 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:11:17.077 DEBUG:teuthology.provision.fog.smithi132:FOG request: GET /task/active None 2023-07-27T14:11:17.080 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:11:22.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:11:30.384 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-27T14:11:36.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:11:37.564 DEBUG:teuthology.provision.fog.smithi120:FOG request: GET /task/active None 2023-07-27T14:11:42.247 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-27T14:11:42.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:11:45.360 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-07-27T14:11:48.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:11:53.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:11:58.260 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-27T14:12:01.328 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-07-27T14:12:04.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:12:09.269 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:12:14.272 WARNING:teuthology.provision.fog.smithi132:[Errno None] Unable to connect to port 22 on 172.21.15.132 2023-07-27T14:12:17.328 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-07-27T14:12:20.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:12:23.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:12:26.384 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-07-27T14:12:32.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:12:55.792 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-07-27T14:13:01.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:13:04.848 WARNING:teuthology.provision.fog.smithi120:[Errno None] Unable to connect to port 22 on 172.21.15.120 2023-07-27T14:13:10.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:13:16.247 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T14:13:16.248 DEBUG:teuthology.orchestra.run.smithi120:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T14:13:20.279 WARNING:teuthology.provision.fog.smithi132:timed out 2023-07-27T14:13:26.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:13:31.623 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-07-27T14:13:31.623 DEBUG:teuthology.orchestra.run.smithi132:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-07-27T14:13:31.897 INFO:teuthology.provision.fog.smithi132:Node is ready 2023-07-27T14:13:31.898 DEBUG:teuthology.orchestra.run.smithi132:> hostname 2023-07-27T14:13:31.913 INFO:teuthology.orchestra.run.smithi132.stdout:smithi132.front.sepia.ceph.com 2023-07-27T14:13:31.913 DEBUG:teuthology.orchestra.run.smithi132:> grep smithi132.front.sepia.ceph.com /etc/hosts 2023-07-27T14:13:31.968 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.15.132 smithi132.front.sepia.ceph.com smithi132 2023-07-27T14:13:31.969 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-07-27T14:13:32.072 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132.front.sepia.ceph.com/smithi132/g' /etc/hosts 2023-07-27T14:13:32.098 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/172.21.15.132/172.21.15.132/g' /etc/hosts 2023-07-27T14:13:32.162 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i -e 's/smithi132.front.sepia.ceph.com/smithi132/g' /etc/hostname 2023-07-27T14:13:32.226 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname smithi132 2023-07-27T14:13:32.289 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-07-27T14:13:32.343 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Red Hat Enterprise Linux" 2023-07-27T14:13:32.343 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8.4 (Ootpa)" 2023-07-27T14:13:32.344 INFO:teuthology.orchestra.run.smithi132.stdout:ID="rhel" 2023-07-27T14:13:32.344 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="fedora" 2023-07-27T14:13:32.344 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8.4" 2023-07-27T14:13:32.344 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2023-07-27T14:13:32.344 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-27T14:13:32.344 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-07-27T14:13:32.345 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-27T14:13:32.345 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.redhat.com/" 2023-07-27T14:13:32.345 INFO:teuthology.orchestra.run.smithi132.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-27T14:13:32.345 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T14:13:32.345 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-27T14:13:32.345 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T14:13:32.346 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-27T14:13:32.346 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-27T14:13:32.346 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-27T14:13:32.346 INFO:teuthology.provision.fog.smithi132:Deploy complete! 2023-07-27T14:13:32.348 DEBUG:teuthology.parallel:result is None 2023-07-27T14:13:36.478 INFO:teuthology.provision.fog.smithi120:Node is ready 2023-07-27T14:13:36.478 DEBUG:teuthology.orchestra.run.smithi120:> hostname 2023-07-27T14:13:36.493 INFO:teuthology.orchestra.run.smithi120.stdout:smithi120 2023-07-27T14:13:36.494 DEBUG:teuthology.orchestra.run.smithi120:> grep smithi120 /etc/hosts 2023-07-27T14:13:36.547 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.15.120 smithi120.front.sepia.ceph.com smithi120 2023-07-27T14:13:36.547 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname smithi120 2023-07-27T14:13:36.702 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/smithi120/smithi120/g' /etc/hosts 2023-07-27T14:13:36.727 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/172.21.15.120/172.21.15.120/g' /etc/hosts 2023-07-27T14:13:36.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i -e 's/smithi120/smithi120/g' /etc/hostname 2023-07-27T14:13:36.855 DEBUG:teuthology.orchestra.run.smithi120:> sudo hostname smithi120 2023-07-27T14:13:36.919 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2023-07-27T14:13:36.973 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux" 2023-07-27T14:13:36.973 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.4 (Ootpa)" 2023-07-27T14:13:36.973 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel" 2023-07-27T14:13:36.974 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora" 2023-07-27T14:13:36.974 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.4" 2023-07-27T14:13:36.974 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8" 2023-07-27T14:13:36.974 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-27T14:13:36.974 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2023-07-27T14:13:36.974 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-27T14:13:36.974 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/" 2023-07-27T14:13:36.975 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-27T14:13:36.975 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T14:13:36.975 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-07-27T14:13:36.975 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T14:13:36.975 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-27T14:13:36.975 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-27T14:13:36.975 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-27T14:13:36.976 INFO:teuthology.provision.fog.smithi120:Deploy complete! 2023-07-27T14:13:36.976 DEBUG:teuthology.parallel:result is None 2023-07-27T14:13:52.134 INFO:teuthology.lock.ops:Checking smithi120.front.sepia.ceph.com 2023-07-27T14:13:52.135 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-27T14:13:57.182 INFO:teuthology.lock.ops:Checking smithi132.front.sepia.ceph.com 2023-07-27T14:13:57.183 INFO:teuthology.lock.ops:New key found. Updating... 2023-07-27T14:14:02.208 INFO:teuthology.lock.ops:Updating [smithi120.front.sepia.ceph.com]: set os type and version on server 2023-07-27T14:14:02.208 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-07-27T14:14:02.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:14:02.276 DEBUG:teuthology.orchestra.run.smithi120:> true 2023-07-27T14:14:02.395 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2023-07-27T14:14:02.395 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2023-07-27T14:14:02.449 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2023-07-27T14:14:02.450 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2023-07-27T14:14:02.504 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Red Hat Enterprise Linux" 2023-07-27T14:14:02.504 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="8.4 (Ootpa)" 2023-07-27T14:14:02.504 INFO:teuthology.orchestra.run.smithi120.stdout:ID="rhel" 2023-07-27T14:14:02.505 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="fedora" 2023-07-27T14:14:02.505 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="8.4" 2023-07-27T14:14:02.505 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el8" 2023-07-27T14:14:02.505 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-27T14:14:02.505 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2023-07-27T14:14:02.505 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-27T14:14:02.505 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.redhat.com/" 2023-07-27T14:14:02.506 INFO:teuthology.orchestra.run.smithi120.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-27T14:14:02.506 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T14:14:02.506 INFO:teuthology.orchestra.run.smithi120.stdout: 2023-07-27T14:14:02.506 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T14:14:02.506 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-27T14:14:02.506 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-27T14:14:02.506 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-27T14:14:02.507 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2023-07-27T14:14:07.548 INFO:teuthology.lock.ops:Updating [smithi132.front.sepia.ceph.com]: set os type and version on server 2023-07-27T14:14:07.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-07-27T14:14:07.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-27T14:14:07.614 DEBUG:teuthology.orchestra.run.smithi132:> true 2023-07-27T14:14:07.731 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2023-07-27T14:14:07.732 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2023-07-27T14:14:07.786 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2023-07-27T14:14:07.787 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2023-07-27T14:14:07.841 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="Red Hat Enterprise Linux" 2023-07-27T14:14:07.842 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="8.4 (Ootpa)" 2023-07-27T14:14:07.842 INFO:teuthology.orchestra.run.smithi132.stdout:ID="rhel" 2023-07-27T14:14:07.842 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="fedora" 2023-07-27T14:14:07.842 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="8.4" 2023-07-27T14:14:07.842 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el8" 2023-07-27T14:14:07.842 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-07-27T14:14:07.842 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2023-07-27T14:14:07.843 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-07-27T14:14:07.843 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://www.redhat.com/" 2023-07-27T14:14:07.843 INFO:teuthology.orchestra.run.smithi132.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-27T14:14:07.843 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-27T14:14:07.843 INFO:teuthology.orchestra.run.smithi132.stdout: 2023-07-27T14:14:07.843 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-27T14:14:07.844 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-07-27T14:14:07.844 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-27T14:14:07.844 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-07-27T14:14:07.845 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2023-07-27T14:14:12.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:14:12.872 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:14:17.979 INFO:teuthology.dispatcher.supervisor:Running job 7354406 2023-07-27T14:14:17.979 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_407880c6d3fb77318fff01c863715090f9c2de69/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354406 --name yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi --description fs/32bits/{begin clusters/fixed-2-ucephfs conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore-ec/bluestore-ec-root overrides/{faked-ino ignorelist_health ignorelist_wrongly_marked_down} tasks/cfuse_workunit_suites_pjd} -- /home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354406/orig.config.yaml 2023-07-27T14:14:17.982 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354406 2023-07-27T14:14:17.983 INFO:teuthology.dispatcher.supervisor:Job PID: 28751 2023-07-27T14:14:17.983 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-07-27T14:16:18.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:16:18.082 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:18:22.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:18:22.165 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:20:26.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:20:26.606 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:22:28.437 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:22:28.437 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:24:28.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:24:28.606 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:26:28.781 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:26:28.782 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:28:33.957 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:28:33.957 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:30:39.133 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:30:39.133 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:32:40.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-27T14:32:40.350 DEBUG:urllib3.util.retry:Converted retries value: 10 -> Retry(total=10, connect=None, read=None, redirect=None, status=None) 2023-07-27T14:32:45.427 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-07-27T14:32:55.534 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-07-27T14:32:55.534 INFO:teuthology.nuke:Checking targets against current locks 2023-07-27T14:33:15.610 DEBUG:teuthology.nuke:shortname: smithi120 2023-07-27T14:33:15.611 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-27T14:33:15.643 DEBUG:teuthology.nuke:shortname: smithi132 2023-07-27T14:33:15.643 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-27T14:33:15.646 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354406', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-27 14:07:50.754149', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6XE3DXQMAAGtH7cTGbEybUUGhKPWqUBcdc3/TuIjJM++j5fZCE3Q6lWMWaS9rjb1dL2Z9tKwAGyWv4Vt4uktLPHwXe1rS5uWuXbl4kKlfBnBH08yj7JMRH02FBlm/Rj1BimhA5YcgS/o8CgncTPXVqAW+JiBTLERChe0/TbwhqZYZ4E4gMqqjtHgjEh8QHz837gFykwXecTf+K2k3ObA9hya3ScdN/LjgZyPASuFyCD3hq0BdKm7F++IUx4I2SD0XqyHyd4HeBpPR8XCbMCSh5rYE3P7soErEi6q/cSsDntVFqS1rbseAJ6Uv+TQgHhM3alyi5Dz8Gfmn0QzT8kgbWEVFQJh2641dbwsv2YWlMwriHN0aspWqbDIcCE59ld9nPnxRfbJWsYrcGQByyAsoJmz3mvjWoCSQKtaVbrNKVJRP78Crjuga85Zj+lDKxHuJXdMQs0Bg4jpJkF/Bdz3wltn58IrBZ20hPlQ2WTAYU+7vyXnyYg6+3oVMx7vlknM='} 2023-07-27T14:33:15.673 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-07-27_03:42:08-fs-wip-yuri10-testing-2023-07-26-0858-pacific-distro-default-smithi/7354406', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-27 14:07:50.756340', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG2WyN1pS1hgz61ehMiZYr2/y0guwuw+tHUDuWB3jQSu9wzERJrsQgdFgir+eh9dAtE3/qupowbBkg2R9aImI8g='} 2023-07-27T14:33:15.699 INFO:teuthology.orchestra.console:Power off smithi120 2023-07-27T14:33:15.699 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T14:33:15.721 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T14:33:15.721 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi120.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:33:15.739 INFO:teuthology.orchestra.console:Power off smithi132 2023-07-27T14:33:15.740 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-27T14:33:15.748 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T14:33:15.759 DEBUG:teuthology.orchestra.console:power off output: Chassis Power Control: Down/Off 2023-07-27T14:33:15.760 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-27T14:33:15.782 DEBUG:teuthology.orchestra.console:check power output: Chassis Power is off 2023-07-27T14:33:15.850 INFO:teuthology.orchestra.console:Power off for smithi120 completed 2023-07-27T14:33:15.884 INFO:teuthology.orchestra.console:Power off for smithi132 completed 2023-07-27T14:33:16.000 INFO:teuthology.lock.ops:unlocked: smithi120.front.sepia.ceph.com 2023-07-27T14:33:16.033 INFO:teuthology.lock.ops:unlocked: smithi132.front.sepia.ceph.com