2023-09-13T23:18:00.206 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-13T23:18:00.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:18:00.357 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb9e9c59670>, signals=[15]) 2023-09-13T23:18:00.358 INFO:teuthology.lock.ops:Start node 'smithi059.front.sepia.ceph.com' reimaging 2023-09-13T23:18:00.359 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: reset os type and version on server 2023-09-13T23:18:00.359 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2023-09-13T23:18:00.378 INFO:teuthology.lock.ops:Start node 'smithi158.front.sepia.ceph.com' reimaging 2023-09-13T23:18:00.378 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: reset os type and version on server 2023-09-13T23:18:00.378 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2023-09-13T23:18:00.380 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /host {"name": "smithi059"} 2023-09-13T23:18:00.403 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /host {"name": "smithi158"} 2023-09-13T23:18:00.456 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-09-13T23:18:00.473 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /image {"name": "smithi_rhel_8.6"} 2023-09-13T23:18:00.499 DEBUG:teuthology.provision.fog.smithi059:Requesting image smithi_rhel_8.6 (ID 5) 2023-09-13T23:18:00.500 DEBUG:teuthology.provision.fog.smithi059:FOG request: PUT /host/164 {"imageID": 5} 2023-09-13T23:18:00.521 DEBUG:teuthology.provision.fog.smithi158:Requesting image smithi_rhel_8.6 (ID 5) 2023-09-13T23:18:00.521 DEBUG:teuthology.provision.fog.smithi158:FOG request: PUT /host/263 {"imageID": 5} 2023-09-13T23:18:00.550 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of rhel 8.6 2023-09-13T23:18:00.550 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:18:00.564 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of rhel 8.6 2023-09-13T23:18:00.564 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:18:00.612 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /tasktype {"name": "deploy"} 2023-09-13T23:18:00.616 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /tasktype {"name": "deploy"} 2023-09-13T23:18:00.649 DEBUG:teuthology.provision.fog.smithi158:FOG request: POST /host/263/task {"taskTypeID": 1} 2023-09-13T23:18:00.651 DEBUG:teuthology.provision.fog.smithi059:FOG request: POST /host/164/task {"taskTypeID": 1} 2023-09-13T23:18:00.719 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:18:00.723 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:18:00.809 INFO:teuthology.orchestra.console.smithi158:Power off 2023-09-13T23:18:00.810 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T23:18:00.822 INFO:teuthology.orchestra.console.smithi059:Power off 2023-09-13T23:18:00.823 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T23:18:00.832 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off 2023-09-13T23:18:00.833 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:00.843 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2023-09-13T23:18:00.843 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:00.854 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-09-13T23:18:00.865 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-09-13T23:18:04.858 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:04.867 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:04.967 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-09-13T23:18:04.974 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-09-13T23:18:08.970 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:08.977 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:09.080 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-09-13T23:18:09.086 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-09-13T23:18:09.181 INFO:teuthology.orchestra.console.smithi158:Power off completed 2023-09-13T23:18:09.282 INFO:teuthology.orchestra.console.smithi158:Power on 2023-09-13T23:18:09.283 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-13T23:18:09.303 DEBUG:teuthology.orchestra.console.smithi158:power on output: Chassis Power Control: Up/On 2023-09-13T23:18:09.303 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:09.324 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-09-13T23:18:13.090 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:13.199 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2023-09-13T23:18:13.300 INFO:teuthology.orchestra.console.smithi059:Power off completed 2023-09-13T23:18:13.325 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:13.402 INFO:teuthology.orchestra.console.smithi059:Power on 2023-09-13T23:18:13.402 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-09-13T23:18:13.423 DEBUG:teuthology.orchestra.console.smithi059:power on output: Chassis Power Control: Up/On 2023-09-13T23:18:13.424 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:13.434 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-09-13T23:18:13.444 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2023-09-13T23:18:17.438 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:17.445 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:18:17.547 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2023-09-13T23:18:17.555 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-09-13T23:18:17.649 INFO:teuthology.orchestra.console.smithi158:Power on completed 2023-09-13T23:18:17.656 INFO:teuthology.orchestra.console.smithi059:Power on completed 2023-09-13T23:18:17.750 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish 2023-09-13T23:18:17.750 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:18:17.758 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2023-09-13T23:18:17.758 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:18:32.822 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:18:32.824 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:18:47.886 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:18:47.887 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:19:02.974 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:19:02.976 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:19:18.050 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:19:18.052 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:19:33.132 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:19:33.133 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:19:48.219 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:19:48.220 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:20:03.302 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:20:03.304 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:20:18.382 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:20:18.384 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:20:33.462 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:20:33.463 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:20:48.534 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:20:48.536 DEBUG:teuthology.provision.fog.smithi059:FOG request: GET /task/active None 2023-09-13T23:20:48.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:21:03.606 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:21:06.970 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2023-09-13T23:21:12.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:21:16.058 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2023-09-13T23:21:18.670 DEBUG:teuthology.provision.fog.smithi158:FOG request: GET /task/active None 2023-09-13T23:21:18.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:21:22.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:21:25.114 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2023-09-13T23:21:31.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:21:34.174 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2023-09-13T23:21:37.182 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:21:40.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:21:43.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:21:46.266 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:21:52.270 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:21:55.322 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:22:01.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:22:04.382 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:22:10.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:22:13.466 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:22:19.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:22:22.522 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:22:28.529 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:22:31.614 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:22:37.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:22:40.182 WARNING:teuthology.provision.fog.smithi059:timed out 2023-09-13T23:22:46.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:22:46.546 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-13T23:22:46.547 DEBUG:teuthology.orchestra.run.smithi059:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-13T23:22:46.860 INFO:teuthology.provision.fog.smithi059:Node is ready 2023-09-13T23:22:46.861 DEBUG:teuthology.orchestra.run.smithi059:> hostname 2023-09-13T23:22:46.885 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2023-09-13T23:22:46.886 DEBUG:teuthology.orchestra.run.smithi059:> grep smithi059.front.sepia.ceph.com /etc/hosts 2023-09-13T23:22:46.944 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2023-09-13T23:22:46.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2023-09-13T23:22:47.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hosts 2023-09-13T23:22:47.110 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/172.21.15.59/172.21.15.59/g' /etc/hosts 2023-09-13T23:22:47.178 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hostname 2023-09-13T23:22:47.247 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2023-09-13T23:22:47.315 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2023-09-13T23:22:47.372 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T23:22:47.372 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T23:22:47.373 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2023-09-13T23:22:47.373 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2023-09-13T23:22:47.373 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.6" 2023-09-13T23:22:47.373 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2023-09-13T23:22:47.373 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T23:22:47.374 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2023-09-13T23:22:47.374 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T23:22:47.374 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T23:22:47.374 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T23:22:47.374 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T23:22:47.375 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-09-13T23:22:47.375 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T23:22:47.375 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T23:22:47.375 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T23:22:47.376 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T23:22:47.376 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2023-09-13T23:22:47.378 DEBUG:teuthology.parallel:result is None 2023-09-13T23:22:56.033 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:23:02.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:23:05.114 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:23:11.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:23:14.170 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2023-09-13T23:23:20.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:23:20.524 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-09-13T23:23:20.525 DEBUG:teuthology.orchestra.run.smithi158:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-09-13T23:23:20.806 INFO:teuthology.provision.fog.smithi158:Node is ready 2023-09-13T23:23:20.806 DEBUG:teuthology.orchestra.run.smithi158:> hostname 2023-09-13T23:23:20.822 INFO:teuthology.orchestra.run.smithi158.stdout:smithi158.front.sepia.ceph.com 2023-09-13T23:23:20.823 DEBUG:teuthology.orchestra.run.smithi158:> grep smithi158.front.sepia.ceph.com /etc/hosts 2023-09-13T23:23:20.878 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.158 smithi158.front.sepia.ceph.com smithi158 2023-09-13T23:23:20.879 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158 2023-09-13T23:23:21.004 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hosts 2023-09-13T23:23:21.031 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/172.21.15.158/172.21.15.158/g' /etc/hosts 2023-09-13T23:23:21.099 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hostname 2023-09-13T23:23:21.165 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158 2023-09-13T23:23:21.230 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2023-09-13T23:23:21.287 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T23:23:21.287 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T23:23:21.287 INFO:teuthology.orchestra.run.smithi158.stdout:ID="rhel" 2023-09-13T23:23:21.288 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="fedora" 2023-09-13T23:23:21.288 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8.6" 2023-09-13T23:23:21.288 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8" 2023-09-13T23:23:21.288 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T23:23:21.289 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2023-09-13T23:23:21.289 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T23:23:21.289 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T23:23:21.289 INFO:teuthology.orchestra.run.smithi158.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T23:23:21.289 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T23:23:21.289 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-09-13T23:23:21.290 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T23:23:21.290 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T23:23:21.290 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T23:23:21.290 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T23:23:21.291 INFO:teuthology.provision.fog.smithi158:Deploy complete! 2023-09-13T23:23:21.291 DEBUG:teuthology.parallel:result is None 2023-09-13T23:23:21.439 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2023-09-13T23:23:21.440 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-13T23:23:21.465 INFO:teuthology.lock.ops:Checking smithi158.front.sepia.ceph.com 2023-09-13T23:23:21.465 INFO:teuthology.lock.ops:New key found. Updating... 2023-09-13T23:23:21.491 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2023-09-13T23:23:21.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2023-09-13T23:23:21.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:23:21.561 DEBUG:teuthology.orchestra.run.smithi059:> true 2023-09-13T23:23:21.696 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2023-09-13T23:23:21.697 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2023-09-13T23:23:21.758 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2023-09-13T23:23:21.759 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2023-09-13T23:23:21.817 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T23:23:21.817 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T23:23:21.818 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2023-09-13T23:23:21.818 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2023-09-13T23:23:21.818 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.6" 2023-09-13T23:23:21.818 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2023-09-13T23:23:21.818 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T23:23:21.819 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2023-09-13T23:23:21.819 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T23:23:21.819 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T23:23:21.819 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T23:23:21.819 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T23:23:21.820 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-09-13T23:23:21.820 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T23:23:21.820 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T23:23:21.820 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T23:23:21.820 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T23:23:21.821 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2023-09-13T23:23:21.840 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: set os type and version on server 2023-09-13T23:23:21.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-09-13T23:23:21.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-13T23:23:21.906 DEBUG:teuthology.orchestra.run.smithi158:> true 2023-09-13T23:23:21.975 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi158.front.sepia.ceph.com' 2023-09-13T23:23:21.976 DEBUG:teuthology.orchestra.run.smithi158:> uname -m 2023-09-13T23:23:22.033 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64 2023-09-13T23:23:22.033 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2023-09-13T23:23:22.090 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="Red Hat Enterprise Linux" 2023-09-13T23:23:22.090 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="8.6 (Ootpa)" 2023-09-13T23:23:22.091 INFO:teuthology.orchestra.run.smithi158.stdout:ID="rhel" 2023-09-13T23:23:22.091 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="fedora" 2023-09-13T23:23:22.091 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="8.6" 2023-09-13T23:23:22.092 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el8" 2023-09-13T23:23:22.092 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-09-13T23:23:22.092 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2023-09-13T23:23:22.092 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-09-13T23:23:22.093 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://www.redhat.com/" 2023-09-13T23:23:22.093 INFO:teuthology.orchestra.run.smithi158.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-09-13T23:23:22.093 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-13T23:23:22.093 INFO:teuthology.orchestra.run.smithi158.stdout: 2023-09-13T23:23:22.094 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-13T23:23:22.094 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-09-13T23:23:22.094 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-09-13T23:23:22.095 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-09-13T23:23:22.095 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2023-09-13T23:23:22.115 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:23:22.221 INFO:teuthology.dispatcher.supervisor:Running job 7394592 2023-09-13T23:23:22.221 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/virtualenv/bin/teuthology -v --owner mchangir --archive /home/teuthworker/archive/mchangir-2023-09-12_05:40:22-fs-wip-mchangir-testing-20230908.140927-testing-default-smithi/7394592 --name mchangir-2023-09-12_05:40:22-fs-wip-mchangir-testing-20230908.140927-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} -- /home/teuthworker/archive/mchangir-2023-09-12_05:40:22-fs-wip-mchangir-testing-20230908.140927-testing-default-smithi/7394592/orig.config.yaml 2023-09-13T23:23:22.225 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/mchangir-2023-09-12_05:40:22-fs-wip-mchangir-testing-20230908.140927-testing-default-smithi/7394592 2023-09-13T23:23:22.226 INFO:teuthology.dispatcher.supervisor:Job PID: 912262 2023-09-13T23:23:22.226 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-09-13T23:25:22.284 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:27:22.443 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:29:22.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:31:22.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:33:22.927 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:35:23.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:37:23.255 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:39:23.423 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-13T23:39:23.496 ERROR:teuthology.dispatcher.supervisor:Child exited with code 120 2023-09-13T23:39:23.604 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-09-13T23:39:23.604 INFO:teuthology.nuke:Checking targets against current locks 2023-09-13T23:39:23.654 DEBUG:teuthology.nuke:shortname: smithi059 2023-09-13T23:39:23.655 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T23:39:23.673 DEBUG:teuthology.nuke:shortname: smithi158 2023-09-13T23:39:23.674 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-13T23:39:23.677 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-09-12_05:40:22-fs-wip-mchangir-testing-20230908.140927-testing-default-smithi/7394592', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 23:17:59.376572', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T23:39:23.699 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi158.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-09-12_05:40:22-fs-wip-mchangir-testing-20230908.140927-testing-default-smithi/7394592', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-13 23:17:59.374647', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-09-13T23:39:23.728 INFO:teuthology.orchestra.console.smithi059:Power off 2023-09-13T23:39:23.729 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T23:39:23.744 INFO:teuthology.orchestra.console.smithi158:Power off 2023-09-13T23:39:23.744 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-13T23:39:23.752 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2023-09-13T23:39:23.752 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:39:23.764 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off 2023-09-13T23:39:23.765 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-13T23:39:23.773 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2023-09-13T23:39:23.786 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2023-09-13T23:39:23.875 INFO:teuthology.orchestra.console.smithi059:Power off completed 2023-09-13T23:39:23.888 INFO:teuthology.orchestra.console.smithi158:Power off completed 2023-09-13T23:39:24.037 INFO:teuthology.lock.ops:unlocked: smithi059.front.sepia.ceph.com 2023-09-13T23:39:24.045 INFO:teuthology.lock.ops:unlocked: smithi158.front.sepia.ceph.com