2023-11-28T09:13:14.626 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T09:13:14.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T09:13:14.789 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7a7ce08dc0>, signals=[15]) 2023-11-28T09:13:14.789 INFO:teuthology.lock.ops:Start node 'smithi103.front.sepia.ceph.com' reimaging 2023-11-28T09:13:14.789 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: reset os type and version on server 2023-11-28T09:13:14.790 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2023-11-28T09:13:14.816 INFO:teuthology.lock.ops:Start node 'smithi122.front.sepia.ceph.com' reimaging 2023-11-28T09:13:14.816 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: reset os type and version on server 2023-11-28T09:13:14.816 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2023-11-28T09:13:14.931 DEBUG:teuthology.provision.fog.smithi103:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-28T09:13:14.958 DEBUG:teuthology.provision.fog.smithi122:Requesting image smithi_rhel_8.6 (ID 5) 2023-11-28T09:13:14.975 INFO:teuthology.provision.fog.smithi103:Scheduling deploy of rhel 8.6 2023-11-28T09:13:15.004 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of rhel 8.6 2023-11-28T09:13:15.232 INFO:teuthology.orchestra.console.smithi103:Power off 2023-11-28T09:13:15.232 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T09:13:15.249 INFO:teuthology.orchestra.console.smithi122:Power off 2023-11-28T09:13:15.250 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T09:13:15.257 DEBUG:teuthology.orchestra.console.smithi103:power off output: Chassis Power Control: Down/Off 2023-11-28T09:13:15.258 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:13:15.271 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2023-11-28T09:13:15.271 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:13:15.283 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2023-11-28T09:13:15.296 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2023-11-28T09:13:15.384 INFO:teuthology.orchestra.console.smithi103:Power off completed 2023-11-28T09:13:15.397 INFO:teuthology.orchestra.console.smithi122:Power off completed 2023-11-28T09:13:15.485 INFO:teuthology.orchestra.console.smithi103:Power on 2023-11-28T09:13:15.485 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-28T09:13:15.497 INFO:teuthology.orchestra.console.smithi122:Power on 2023-11-28T09:13:15.498 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-11-28T09:13:15.517 DEBUG:teuthology.orchestra.console.smithi103:power on output: Chassis Power Control: Up/On 2023-11-28T09:13:15.517 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:13:15.529 DEBUG:teuthology.orchestra.console.smithi122:power on output: Chassis Power Control: Up/On 2023-11-28T09:13:15.529 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:13:15.543 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2023-11-28T09:13:15.556 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2023-11-28T09:13:19.546 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:13:19.557 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:13:19.660 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2023-11-28T09:13:19.669 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2023-11-28T09:13:19.769 INFO:teuthology.orchestra.console.smithi122:Power on completed 2023-11-28T09:13:19.870 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2023-11-28T09:13:23.662 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:13:23.772 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2023-11-28T09:13:23.872 INFO:teuthology.orchestra.console.smithi103:Power on completed 2023-11-28T09:13:23.973 INFO:teuthology.provision.fog.smithi103:Waiting for deploy to finish 2023-11-28T09:16:05.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:08.898 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-11-28T09:16:14.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:17.954 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-11-28T09:16:23.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:24.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:27.042 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-11-28T09:16:28.002 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2023-11-28T09:16:33.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:34.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:36.098 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-11-28T09:16:37.058 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2023-11-28T09:16:42.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:43.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:45.190 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-11-28T09:16:46.114 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2023-11-28T09:16:51.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:52.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:16:54.274 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2023-11-28T09:16:55.174 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2023-11-28T09:17:00.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:17:01.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:17:04.258 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2023-11-28T09:17:10.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:17:13.314 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2023-11-28T09:17:19.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:17:22.370 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2023-11-28T09:17:28.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:18:00.303 WARNING:teuthology.provision.fog.smithi122:timed out 2023-11-28T09:18:06.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:18:06.634 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-28T09:18:06.635 DEBUG:teuthology.orchestra.run.smithi122:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-28T09:18:06.937 INFO:teuthology.provision.fog.smithi122:Node is ready 2023-11-28T09:18:06.938 DEBUG:teuthology.orchestra.run.smithi122:> hostname 2023-11-28T09:18:06.957 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 2023-11-28T09:18:06.957 DEBUG:teuthology.orchestra.run.smithi122:> grep smithi122 /etc/hosts 2023-11-28T09:18:07.017 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122 smithi122.front.sepia.ceph.com smithi122 2023-11-28T09:18:07.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122 2023-11-28T09:18:07.141 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi122/smithi122/g' /etc/hosts 2023-11-28T09:18:07.172 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/172.21.15.122/172.21.15.122/g' /etc/hosts 2023-11-28T09:18:07.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi122/smithi122/g' /etc/hostname 2023-11-28T09:18:07.310 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122 2023-11-28T09:18:07.377 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2023-11-28T09:18:07.434 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T09:18:07.434 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T09:18:07.434 INFO:teuthology.orchestra.run.smithi122.stdout:ID="rhel" 2023-11-28T09:18:07.434 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="fedora" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="8.6" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el8" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout: 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T09:18:07.435 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T09:18:07.436 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T09:18:07.436 INFO:teuthology.provision.fog.smithi122:Deploy complete! 2023-11-28T09:18:07.437 DEBUG:teuthology.parallel:result is None 2023-11-28T09:18:28.400 WARNING:teuthology.provision.fog.smithi103:timed out 2023-11-28T09:18:34.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:18:34.623 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-28T09:18:34.623 DEBUG:teuthology.orchestra.run.smithi103:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-11-28T09:18:34.904 INFO:teuthology.provision.fog.smithi103:Node is ready 2023-11-28T09:18:34.904 DEBUG:teuthology.orchestra.run.smithi103:> hostname 2023-11-28T09:18:34.923 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 2023-11-28T09:18:34.923 DEBUG:teuthology.orchestra.run.smithi103:> grep smithi103 /etc/hosts 2023-11-28T09:18:34.982 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103 smithi103.front.sepia.ceph.com smithi103 2023-11-28T09:18:34.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname smithi103 2023-11-28T09:18:35.104 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/smithi103/smithi103/g' /etc/hosts 2023-11-28T09:18:35.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/172.21.15.103/172.21.15.103/g' /etc/hosts 2023-11-28T09:18:35.201 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/smithi103/smithi103/g' /etc/hostname 2023-11-28T09:18:35.269 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname smithi103 2023-11-28T09:18:35.335 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2023-11-28T09:18:35.391 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T09:18:35.391 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:ID="rhel" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="fedora" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="8.6" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el8" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T09:18:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-11-28T09:18:35.393 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T09:18:35.393 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T09:18:35.393 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T09:18:35.393 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T09:18:35.393 INFO:teuthology.provision.fog.smithi103:Deploy complete! 2023-11-28T09:18:35.394 DEBUG:teuthology.parallel:result is None 2023-11-28T09:18:35.539 INFO:teuthology.lock.ops:Checking smithi103.front.sepia.ceph.com 2023-11-28T09:18:35.540 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com 2023-11-28T09:18:35.541 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: set os type and version on server 2023-11-28T09:18:35.541 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2023-11-28T09:18:35.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:18:35.609 DEBUG:teuthology.orchestra.run.smithi103:> true 2023-11-28T09:18:35.674 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2023-11-28T09:18:35.674 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2023-11-28T09:18:35.730 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2023-11-28T09:18:35.731 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2023-11-28T09:18:35.786 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:ID="rhel" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="fedora" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="8.6" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el8" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T09:18:35.787 INFO:teuthology.orchestra.run.smithi103.stdout: 2023-11-28T09:18:35.788 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T09:18:35.788 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T09:18:35.788 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T09:18:35.788 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T09:18:35.788 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2023-11-28T09:18:35.819 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server 2023-11-28T09:18:35.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2023-11-28T09:18:35.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T09:18:35.886 DEBUG:teuthology.orchestra.run.smithi122:> true 2023-11-28T09:18:36.026 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2023-11-28T09:18:36.026 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2023-11-28T09:18:36.085 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2023-11-28T09:18:36.086 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2023-11-28T09:18:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T09:18:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T09:18:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:ID="rhel" 2023-11-28T09:18:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="fedora" 2023-11-28T09:18:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="8.6" 2023-11-28T09:18:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el8" 2023-11-28T09:18:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T09:18:36.144 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout: 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T09:18:36.145 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T09:18:36.146 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2023-11-28T09:18:36.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T09:18:36.259 INFO:teuthology.dispatcher.supervisor:Running job 7469888 2023-11-28T09:18:36.259 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469888 --name rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/multimds_misc} -- /home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469888/orig.config.yaml 2023-11-28T09:18:36.262 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469888 2023-11-28T09:18:36.262 INFO:teuthology.dispatcher.supervisor:Job PID: 575255 2023-11-28T09:18:36.263 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-11-28T09:20:36.364 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T09:22:36.535 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T09:24:36.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T09:26:36.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T09:28:37.035 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T09:30:37.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T09:30:37.276 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-11-28T09:30:37.386 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-11-28T09:30:37.386 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T09:30:37.438 DEBUG:teuthology.nuke:shortname: smithi103 2023-11-28T09:30:37.439 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T09:30:37.462 DEBUG:teuthology.nuke:shortname: smithi122 2023-11-28T09:30:37.462 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T09:30:37.466 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469888', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 09:13:13.762545', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T09:30:37.488 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-28_06:34:49-fs-rishabh-2023nov26-testing-default-smithi/7469888', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 09:13:13.761094', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T09:30:37.518 INFO:teuthology.orchestra.console.smithi103:Power off 2023-11-28T09:30:37.519 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T09:30:37.540 DEBUG:teuthology.orchestra.console.smithi103:power off output: Chassis Power Control: Down/Off 2023-11-28T09:30:37.540 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:30:37.556 INFO:teuthology.orchestra.console.smithi122:Power off 2023-11-28T09:30:37.557 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T09:30:37.568 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2023-11-28T09:30:37.583 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2023-11-28T09:30:37.583 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T09:30:37.603 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2023-11-28T09:30:37.669 INFO:teuthology.orchestra.console.smithi103:Power off completed 2023-11-28T09:30:37.705 INFO:teuthology.orchestra.console.smithi122:Power off completed 2023-11-28T09:30:37.825 INFO:teuthology.lock.ops:unlocked: smithi103.front.sepia.ceph.com 2023-11-28T09:30:37.850 INFO:teuthology.lock.ops:unlocked: smithi122.front.sepia.ceph.com