2023-12-22T08:12:12.811 INFO:root:teuthology version: 0.0.1.dev226+g9680d2a 2023-12-22T08:12:12.812 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T08:12:12.955 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa2529a7940>, signals=[15]) 2023-12-22T08:12:12.956 INFO:teuthology.lock.ops:Start node 'smithi087.front.sepia.ceph.com' reimaging 2023-12-22T08:12:12.956 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: reset os type and version on server 2023-12-22T08:12:12.956 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-12-22T08:12:12.974 INFO:teuthology.lock.ops:Start node 'smithi129.front.sepia.ceph.com' reimaging 2023-12-22T08:12:12.975 INFO:teuthology.lock.ops:Updating [smithi129.front.sepia.ceph.com]: reset os type and version on server 2023-12-22T08:12:12.975 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2023-12-22T08:12:13.204 DEBUG:teuthology.provision.fog.smithi129:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-22T08:12:13.211 DEBUG:teuthology.provision.fog.smithi087:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-22T08:12:13.279 INFO:teuthology.provision.fog.smithi129:Scheduling deploy of rhel 8.6 2023-12-22T08:12:13.348 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of rhel 8.6 2023-12-22T08:12:14.111 INFO:teuthology.orchestra.console.smithi087:Power off 2023-12-22T08:12:14.112 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T08:12:14.126 INFO:teuthology.orchestra.console.smithi129:Power off 2023-12-22T08:12:14.126 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T08:12:14.133 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2023-12-22T08:12:14.133 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:14.151 DEBUG:teuthology.orchestra.console.smithi129:power off output: Chassis Power Control: Down/Off 2023-12-22T08:12:14.152 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:14.158 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-12-22T08:12:14.172 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2023-12-22T08:12:18.158 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:18.173 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:18.267 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-12-22T08:12:18.282 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2023-12-22T08:12:22.270 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:22.283 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:22.378 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-12-22T08:12:22.391 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2023-12-22T08:12:22.479 INFO:teuthology.orchestra.console.smithi087:Power off completed 2023-12-22T08:12:22.492 INFO:teuthology.orchestra.console.smithi129:Power off completed 2023-12-22T08:12:22.580 INFO:teuthology.orchestra.console.smithi087:Power on 2023-12-22T08:12:22.580 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-22T08:12:22.594 INFO:teuthology.orchestra.console.smithi129:Power on 2023-12-22T08:12:22.594 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-22T08:12:22.602 DEBUG:teuthology.orchestra.console.smithi087:power on output: Chassis Power Control: Up/On 2023-12-22T08:12:22.602 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:22.613 DEBUG:teuthology.orchestra.console.smithi129:power on output: Chassis Power Control: Up/On 2023-12-22T08:12:22.614 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:22.623 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-12-22T08:12:22.633 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2023-12-22T08:12:26.628 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:26.634 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:26.736 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-12-22T08:12:26.741 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2023-12-22T08:12:30.738 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:30.745 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:12:30.847 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2023-12-22T08:12:30.853 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2023-12-22T08:12:30.948 INFO:teuthology.orchestra.console.smithi087:Power on completed 2023-12-22T08:12:30.954 INFO:teuthology.orchestra.console.smithi129:Power on completed 2023-12-22T08:12:31.049 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish 2023-12-22T08:12:31.054 INFO:teuthology.provision.fog.smithi129:Waiting for deploy to finish 2023-12-22T08:18:04.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:18:07.730 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:18:13.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:18:16.820 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:18:22.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:18:25.873 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:18:31.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:18:34.929 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:18:40.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:18:43.988 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:18:49.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:18:53.073 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:18:59.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:07.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:10.418 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:19:16.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:17.649 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:19:19.506 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:19:23.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:25.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:26.706 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:19:28.562 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:19:32.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:34.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:35.761 WARNING:teuthology.provision.fog.smithi129:[Errno None] Unable to connect to port 22 on 172.21.15.129 2023-12-22T08:19:37.617 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:19:41.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:42.194 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-22T08:19:42.195 DEBUG:teuthology.orchestra.run.smithi129:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-22T08:19:43.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:46.673 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:19:52.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:19:55.729 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:19:58.142 INFO:teuthology.provision.fog.smithi129:Node is ready 2023-12-22T08:19:58.142 DEBUG:teuthology.orchestra.run.smithi129:> hostname 2023-12-22T08:19:58.163 INFO:teuthology.orchestra.run.smithi129.stdout:smithi045 2023-12-22T08:19:58.163 DEBUG:teuthology.orchestra.run.smithi129:> grep smithi045 /etc/hosts 2023-12-22T08:19:58.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-22T08:19:58.224 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i -e 's/smithi045/smithi129/g' /etc/hostname 2023-12-22T08:19:58.344 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname smithi129 2023-12-22T08:19:58.372 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2023-12-22T08:19:58.430 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T08:19:58.430 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="8.6 (Ootpa)" 2023-12-22T08:19:58.430 INFO:teuthology.orchestra.run.smithi129.stdout:ID="rhel" 2023-12-22T08:19:58.430 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="fedora" 2023-12-22T08:19:58.430 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="8.6" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el8" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout: 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T08:19:58.431 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-22T08:19:58.432 INFO:teuthology.provision.fog.smithi129:Deploy complete! 2023-12-22T08:19:58.433 DEBUG:teuthology.parallel:result is None 2023-12-22T08:20:01.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:20:04.785 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:20:10.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:20:21.138 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:20:27.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:20:30.197 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:20:36.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:20:39.281 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2023-12-22T08:20:45.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:20:47.099 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-22T08:20:47.099 DEBUG:teuthology.orchestra.run.smithi087:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-22T08:21:07.334 INFO:teuthology.provision.fog.smithi087:Node is ready 2023-12-22T08:21:07.334 DEBUG:teuthology.orchestra.run.smithi087:> hostname 2023-12-22T08:21:07.356 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com 2023-12-22T08:21:07.356 DEBUG:teuthology.orchestra.run.smithi087:> grep smithi087.front.sepia.ceph.com /etc/hosts 2023-12-22T08:21:07.416 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2023-12-22T08:21:07.417 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2023-12-22T08:21:07.531 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087.front.sepia.ceph.com/smithi087/g' /etc/hosts 2023-12-22T08:21:07.561 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/172.21.15.87/172.21.15.87/g' /etc/hosts 2023-12-22T08:21:07.630 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087.front.sepia.ceph.com/smithi087/g' /etc/hostname 2023-12-22T08:21:07.699 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2023-12-22T08:21:07.765 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-12-22T08:21:07.822 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.6 (Ootpa)" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.6" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-12-22T08:21:07.823 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T08:21:07.824 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-22T08:21:07.824 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T08:21:07.824 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-22T08:21:07.824 INFO:teuthology.provision.fog.smithi087:Deploy complete! 2023-12-22T08:21:07.824 DEBUG:teuthology.parallel:result is None 2023-12-22T08:21:07.976 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com 2023-12-22T08:21:07.977 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-22T08:21:08.011 INFO:teuthology.lock.ops:Checking smithi129.front.sepia.ceph.com 2023-12-22T08:21:08.011 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-22T08:21:08.028 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server 2023-12-22T08:21:08.028 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-12-22T08:21:08.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:21:08.096 DEBUG:teuthology.orchestra.run.smithi087:> true 2023-12-22T08:21:08.119 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2023-12-22T08:21:08.119 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2023-12-22T08:21:08.175 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2023-12-22T08:21:08.175 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.6 (Ootpa)" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.6" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout: 2023-12-22T08:21:08.232 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T08:21:08.233 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-22T08:21:08.233 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T08:21:08.233 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-22T08:21:08.233 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2023-12-22T08:21:08.252 INFO:teuthology.lock.ops:Updating [smithi129.front.sepia.ceph.com]: set os type and version on server 2023-12-22T08:21:08.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2023-12-22T08:21:08.253 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-22T08:21:08.319 DEBUG:teuthology.orchestra.run.smithi129:> true 2023-12-22T08:21:08.459 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2023-12-22T08:21:08.459 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2023-12-22T08:21:08.518 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2023-12-22T08:21:08.518 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="Red Hat Enterprise Linux" 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="8.6 (Ootpa)" 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:ID="rhel" 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="fedora" 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="8.6" 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el8" 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2023-12-22T08:21:08.576 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-22T08:21:08.577 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://www.redhat.com/" 2023-12-22T08:21:08.577 INFO:teuthology.orchestra.run.smithi129.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-22T08:21:08.577 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-22T08:21:08.577 INFO:teuthology.orchestra.run.smithi129.stdout: 2023-12-22T08:21:08.577 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-22T08:21:08.577 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-22T08:21:08.577 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-22T08:21:08.577 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-22T08:21:08.578 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2023-12-22T08:21:08.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T08:21:08.692 INFO:teuthology.dispatcher.supervisor:Running job 7499314 2023-12-22T08:21:08.692 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9680d2aba1e76f64ade295c81599c97907457848/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-12-22_08:10:43-fs-rishabh-mdscap-reef-testing-default-smithi/7499314 --name rishabh-2023-12-22_08:10:43-fs-rishabh-mdscap-reef-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/scrub} -- /home/teuthworker/archive/rishabh-2023-12-22_08:10:43-fs-rishabh-mdscap-reef-testing-default-smithi/7499314/orig.config.yaml 2023-12-22T08:21:08.695 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-12-22_08:10:43-fs-rishabh-mdscap-reef-testing-default-smithi/7499314 2023-12-22T08:21:08.696 INFO:teuthology.dispatcher.supervisor:Job PID: 1071827 2023-12-22T08:21:08.696 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-22T08:23:08.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T08:25:08.969 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T08:27:09.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T08:29:09.306 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T08:31:09.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-22T08:31:09.569 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-22T08:31:09.689 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-22T08:31:09.689 INFO:teuthology.nuke:Checking targets against current locks 2023-12-22T08:31:09.744 DEBUG:teuthology.nuke:shortname: smithi087 2023-12-22T08:31:09.744 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-22T08:31:09.773 DEBUG:teuthology.nuke:shortname: smithi129 2023-12-22T08:31:09.773 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-22T08:31:09.777 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-22_08:10:43-fs-rishabh-mdscap-reef-testing-default-smithi/7499314', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-22 08:12:11.948641', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI405syIq+C15tDnmq7P/kqeUytlfugufIs24VLVWnfOliZEdqZPQxZkVlyuxHQhYLbqeWy4uXAMH2az0H6q1u8='} 2023-12-22T08:31:09.807 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-22_08:10:43-fs-rishabh-mdscap-reef-testing-default-smithi/7499314', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-22 08:12:11.947245', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/AuxRtJSiiWAVE/uJCGhqE70fzcAN/kOrCPz/rI3iEeBqGcaLqKzmZw2N1PH1Uikmw3Q40kec4wKfMu8eGlHA='} 2023-12-22T08:31:09.840 INFO:teuthology.orchestra.console.smithi087:Power off 2023-12-22T08:31:09.841 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T08:31:09.876 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2023-12-22T08:31:09.876 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:31:09.891 INFO:teuthology.orchestra.console.smithi129:Power off 2023-12-22T08:31:09.892 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-22T08:31:09.904 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2023-12-22T08:31:09.914 DEBUG:teuthology.orchestra.console.smithi129:power off output: Chassis Power Control: Down/Off 2023-12-22T08:31:09.914 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-22T08:31:09.940 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2023-12-22T08:31:10.006 INFO:teuthology.orchestra.console.smithi087:Power off completed 2023-12-22T08:31:10.041 INFO:teuthology.orchestra.console.smithi129:Power off completed 2023-12-22T08:31:10.169 INFO:teuthology.lock.ops:unlocked: smithi087.front.sepia.ceph.com 2023-12-22T08:31:10.192 INFO:teuthology.lock.ops:unlocked: smithi129.front.sepia.ceph.com