2023-09-29T17:34:54.519 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-29T17:34:54.520 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:34:54.661 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcc6fdab670>, signals=[15]) 2023-09-29T17:34:54.662 INFO:teuthology.lock.ops:Start node 'smithi031.front.sepia.ceph.com' reimaging 2023-09-29T17:34:54.662 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: reset os type and version on server 2023-09-29T17:34:54.662 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-09-29T17:34:54.684 INFO:teuthology.lock.ops:Start node 'smithi155.front.sepia.ceph.com' reimaging 2023-09-29T17:34:54.684 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: reset os type and version on server 2023-09-29T17:34:54.684 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-09-29T17:34:54.685 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /host {"name": "smithi031"} 2023-09-29T17:34:54.709 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /host {"name": "smithi155"} 2023-09-29T17:34:54.766 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-09-29T17:34:54.777 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /image {"name": "smithi_rhel_8.4"} 2023-09-29T17:34:54.805 DEBUG:teuthology.provision.fog.smithi031:Requesting image smithi_rhel_8.4 (ID 7) 2023-09-29T17:34:54.805 DEBUG:teuthology.provision.fog.smithi031:FOG request: PUT /host/136 {"imageID": 7} 2023-09-29T17:34:54.833 DEBUG:teuthology.provision.fog.smithi155:Requesting image smithi_rhel_8.4 (ID 7) 2023-09-29T17:34:54.833 DEBUG:teuthology.provision.fog.smithi155:FOG request: PUT /host/260 {"imageID": 7} 2023-09-29T17:34:54.857 INFO:teuthology.provision.fog.smithi031:Scheduling deploy of rhel 8.4 2023-09-29T17:34:54.857 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:34:54.888 INFO:teuthology.provision.fog.smithi155:Scheduling deploy of rhel 8.4 2023-09-29T17:34:54.889 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:34:54.929 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /tasktype {"name": "deploy"} 2023-09-29T17:34:54.952 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /tasktype {"name": "deploy"} 2023-09-29T17:34:54.966 DEBUG:teuthology.provision.fog.smithi031:FOG request: POST /host/136/task {"taskTypeID": 1} 2023-09-29T17:34:54.987 DEBUG:teuthology.provision.fog.smithi155:FOG request: POST /host/260/task {"taskTypeID": 1} 2023-09-29T17:34:55.021 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:34:55.041 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:34:55.133 INFO:teuthology.orchestra.console.smithi031:Power off 2023-09-29T17:34:55.134 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T17:34:55.150 INFO:teuthology.orchestra.console.smithi155:Power off 2023-09-29T17:34:55.150 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T17:34:55.159 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-09-29T17:34:55.159 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:34:55.175 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2023-09-29T17:34:55.175 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:34:55.190 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-09-29T17:34:55.196 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-09-29T17:34:59.194 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:34:59.202 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:34:59.304 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-09-29T17:34:59.312 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-09-29T17:35:03.306 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:35:03.314 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:35:03.415 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-09-29T17:35:03.421 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-09-29T17:35:03.517 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-09-29T17:35:03.617 INFO:teuthology.orchestra.console.smithi031:Power on 2023-09-29T17:35:03.618 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-29T17:35:03.640 DEBUG:teuthology.orchestra.console.smithi031:power on output: Chassis Power Control: Up/On 2023-09-29T17:35:03.640 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:35:03.663 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-09-29T17:35:07.426 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:35:07.535 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-09-29T17:35:07.636 INFO:teuthology.orchestra.console.smithi155:Power off completed 2023-09-29T17:35:07.664 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:35:07.737 INFO:teuthology.orchestra.console.smithi155:Power on 2023-09-29T17:35:07.737 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-29T17:35:07.759 DEBUG:teuthology.orchestra.console.smithi155:power on output: Chassis Power Control: Up/On 2023-09-29T17:35:07.759 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:35:07.773 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-09-29T17:35:07.781 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-09-29T17:35:07.874 INFO:teuthology.orchestra.console.smithi031:Power on completed 2023-09-29T17:35:07.974 INFO:teuthology.provision.fog.smithi031:Waiting for deploy to finish 2023-09-29T17:35:07.975 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:35:11.786 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:35:11.895 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-09-29T17:35:15.898 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T17:35:16.008 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-09-29T17:35:16.109 INFO:teuthology.orchestra.console.smithi155:Power on completed 2023-09-29T17:35:16.210 INFO:teuthology.provision.fog.smithi155:Waiting for deploy to finish 2023-09-29T17:35:16.210 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:35:23.070 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:35:31.290 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:35:38.151 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:35:46.374 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:35:53.238 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:36:01.454 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:36:08.312 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:36:16.530 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:36:23.388 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:36:31.598 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:36:38.478 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:36:46.666 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:36:53.547 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:37:01.738 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:37:08.614 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:37:16.802 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:37:23.682 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:37:31.870 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:37:38.758 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:37:46.950 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:37:53.822 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:38:02.014 DEBUG:teuthology.provision.fog.smithi155:FOG request: GET /task/active None 2023-09-29T17:38:02.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:05.146 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-29T17:38:08.883 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:38:11.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:14.206 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-29T17:38:20.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:23.290 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-29T17:38:23.942 DEBUG:teuthology.provision.fog.smithi031:FOG request: GET /task/active None 2023-09-29T17:38:24.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:27.066 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-09-29T17:38:29.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:32.350 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-29T17:38:33.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:36.126 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-09-29T17:38:38.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:41.434 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-29T17:38:42.129 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:45.178 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-09-29T17:38:47.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:50.490 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-29T17:38:51.180 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:54.234 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-09-29T17:38:56.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:38:59.546 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-29T17:39:00.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:39:03.294 WARNING:teuthology.provision.fog.smithi031:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-09-29T17:39:05.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:39:08.602 WARNING:teuthology.provision.fog.smithi155:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-09-29T17:39:09.297 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:39:14.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:40:09.350 WARNING:teuthology.provision.fog.smithi031:timed out 2023-09-29T17:40:14.610 WARNING:teuthology.provision.fog.smithi155:timed out 2023-09-29T17:40:15.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:40:15.704 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-29T17:40:15.705 DEBUG:teuthology.orchestra.run.smithi031:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-29T17:40:15.961 INFO:teuthology.provision.fog.smithi031:Node is ready 2023-09-29T17:40:15.961 DEBUG:teuthology.orchestra.run.smithi031:> hostname 2023-09-29T17:40:15.977 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031.front.sepia.ceph.com 2023-09-29T17:40:15.977 DEBUG:teuthology.orchestra.run.smithi031:> grep smithi031.front.sepia.ceph.com /etc/hosts 2023-09-29T17:40:16.034 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31 smithi031.front.sepia.ceph.com smithi031 2023-09-29T17:40:16.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname smithi031 2023-09-29T17:40:16.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/smithi031.front.sepia.ceph.com/smithi031/g' /etc/hosts 2023-09-29T17:40:16.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/172.21.15.31/172.21.15.31/g' /etc/hosts 2023-09-29T17:40:16.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i -e 's/smithi031.front.sepia.ceph.com/smithi031/g' /etc/hostname 2023-09-29T17:40:16.310 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname smithi031 2023-09-29T17:40:16.375 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-09-29T17:40:16.430 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T17:40:16.430 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T17:40:16.430 INFO:teuthology.orchestra.run.smithi031.stdout:ID="rhel" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="fedora" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8.4" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T17:40:16.431 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T17:40:16.432 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T17:40:16.432 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T17:40:16.432 INFO:teuthology.provision.fog.smithi031:Deploy complete! 2023-09-29T17:40:16.433 DEBUG:teuthology.parallel:result is None 2023-09-29T17:40:20.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:40:20.958 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-29T17:40:20.958 DEBUG:teuthology.orchestra.run.smithi155:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-29T17:40:21.229 INFO:teuthology.provision.fog.smithi155:Node is ready 2023-09-29T17:40:21.229 DEBUG:teuthology.orchestra.run.smithi155:> hostname 2023-09-29T17:40:21.244 INFO:teuthology.orchestra.run.smithi155.stdout:smithi155.front.sepia.ceph.com 2023-09-29T17:40:21.244 DEBUG:teuthology.orchestra.run.smithi155:> grep smithi155.front.sepia.ceph.com /etc/hosts 2023-09-29T17:40:21.299 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.15.155 smithi155.front.sepia.ceph.com smithi155 2023-09-29T17:40:21.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname smithi155 2023-09-29T17:40:21.425 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi155.front.sepia.ceph.com/smithi155/g' /etc/hosts 2023-09-29T17:40:21.451 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/172.21.15.155/172.21.15.155/g' /etc/hosts 2023-09-29T17:40:21.516 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i -e 's/smithi155.front.sepia.ceph.com/smithi155/g' /etc/hostname 2023-09-29T17:40:21.580 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname smithi155 2023-09-29T17:40:21.643 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-09-29T17:40:21.698 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T17:40:21.698 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T17:40:21.699 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel" 2023-09-29T17:40:21.699 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora" 2023-09-29T17:40:21.699 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.4" 2023-09-29T17:40:21.699 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2023-09-29T17:40:21.699 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T17:40:21.700 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-09-29T17:40:21.700 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T17:40:21.700 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T17:40:21.700 INFO:teuthology.orchestra.run.smithi155.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T17:40:21.700 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T17:40:21.701 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-09-29T17:40:21.701 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T17:40:21.701 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T17:40:21.701 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T17:40:21.701 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T17:40:21.702 INFO:teuthology.provision.fog.smithi155:Deploy complete! 2023-09-29T17:40:21.702 DEBUG:teuthology.parallel:result is None 2023-09-29T17:40:21.851 INFO:teuthology.lock.ops:Checking smithi031.front.sepia.ceph.com 2023-09-29T17:40:21.851 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-29T17:40:21.879 INFO:teuthology.lock.ops:Checking smithi155.front.sepia.ceph.com 2023-09-29T17:40:21.879 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-29T17:40:21.897 INFO:teuthology.lock.ops:Updating [smithi031.front.sepia.ceph.com]: set os type and version on server 2023-09-29T17:40:21.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-09-29T17:40:21.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:40:21.966 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-09-29T17:40:22.039 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-09-29T17:40:22.039 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-09-29T17:40:22.097 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-09-29T17:40:22.097 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-09-29T17:40:22.155 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T17:40:22.155 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T17:40:22.155 INFO:teuthology.orchestra.run.smithi031.stdout:ID="rhel" 2023-09-29T17:40:22.155 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="fedora" 2023-09-29T17:40:22.156 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8.4" 2023-09-29T17:40:22.156 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2023-09-29T17:40:22.156 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T17:40:22.156 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-09-29T17:40:22.156 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T17:40:22.156 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T17:40:22.157 INFO:teuthology.orchestra.run.smithi031.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T17:40:22.157 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T17:40:22.157 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-09-29T17:40:22.157 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T17:40:22.157 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T17:40:22.157 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T17:40:22.157 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T17:40:22.158 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-09-29T17:40:22.180 INFO:teuthology.lock.ops:Updating [smithi155.front.sepia.ceph.com]: set os type and version on server 2023-09-29T17:40:22.180 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-09-29T17:40:22.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T17:40:22.246 DEBUG:teuthology.orchestra.run.smithi155:> true 2023-09-29T17:40:22.308 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-09-29T17:40:22.309 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2023-09-29T17:40:22.363 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2023-09-29T17:40:22.363 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-09-29T17:40:22.418 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Red Hat Enterprise Linux" 2023-09-29T17:40:22.418 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="8.4 (Ootpa)" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:ID="rhel" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="fedora" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="8.4" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el8" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.redhat.com/" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-29T17:40:22.419 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-29T17:40:22.420 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-09-29T17:40:22.420 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-29T17:40:22.420 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-09-29T17:40:22.420 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-29T17:40:22.420 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-09-29T17:40:22.420 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-09-29T17:40:22.447 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:40:22.545 INFO:teuthology.dispatcher.supervisor:Running job 7405782 2023-09-29T17:40:22.545 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-09-28_22:58:30-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7405782 --name yuriw-2023-09-28_22:58:30-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/admin} -- /home/teuthworker/archive/yuriw-2023-09-28_22:58:30-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7405782/orig.config.yaml 2023-09-29T17:40:22.549 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-09-28_22:58:30-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7405782 2023-09-29T17:40:22.549 INFO:teuthology.dispatcher.supervisor:Job PID: 2903866 2023-09-29T17:40:22.549 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-29T17:42:22.650 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:44:22.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:46:22.979 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:48:23.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:50:23.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:52:23.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:54:23.658 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:56:23.823 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T17:58:23.984 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:00:24.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:02:24.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:04:24.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:06:24.663 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:08:24.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:10:25.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:12:25.171 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:14:25.327 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:16:25.500 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:18:25.673 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:20:25.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:22:26.067 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:24:26.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:26:26.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:28:26.571 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:30:26.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:32:26.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:34:27.071 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:36:27.234 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:38:27.395 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:40:27.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:42:27.732 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T18:42:27.805 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-09-29T18:42:27.925 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-29T18:42:27.925 INFO:teuthology.nuke:Checking targets against current locks 2023-09-29T18:42:27.971 DEBUG:teuthology.nuke:shortname: smithi031 2023-09-29T18:42:27.971 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T18:42:27.994 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-28_22:58:30-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7405782', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 17:34:53.713143', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPDnNi8+x+Hlrd4wD18F8DThYzYaG1ibI5e6N0H1qduH+3ya7xERS7eoV91deKfrx0ELNtySxUqduIKUyfThMEQ='} 2023-09-29T18:42:27.999 DEBUG:teuthology.nuke:shortname: smithi155 2023-09-29T18:42:27.999 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T18:42:28.019 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-28_22:58:30-fs-wip-yuri2-testing-2023-09-27-0908-quincy-distro-default-smithi/7405782', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 17:34:53.714889', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHvt4lWhkxzpmIrwoZ3XJxZ0Pk1XD31/6KO6f98/5wFWqVQ68m4WOgIH56kJHKnbJuAblhKxXe/SYa5anpAWig='} 2023-09-29T18:42:28.047 INFO:teuthology.orchestra.console.smithi031:Power off 2023-09-29T18:42:28.047 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T18:42:28.070 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-09-29T18:42:28.070 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T18:42:28.086 INFO:teuthology.orchestra.console.smithi155:Power off 2023-09-29T18:42:28.086 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T18:42:28.093 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-09-29T18:42:28.107 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2023-09-29T18:42:28.108 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T18:42:28.128 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-09-29T18:42:28.195 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-09-29T18:42:28.229 INFO:teuthology.orchestra.console.smithi155:Power off completed 2023-09-29T18:42:28.355 INFO:teuthology.lock.ops:unlocked: smithi031.front.sepia.ceph.com 2023-09-29T18:42:28.376 INFO:teuthology.lock.ops:unlocked: smithi155.front.sepia.ceph.com