2023-12-19T13:09:54.541 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-19T13:09:54.541 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T13:09:54.888 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7c01c38940>, signals=[15]) 2023-12-19T13:09:54.888 INFO:teuthology.lock.ops:Start node 'smithi029.front.sepia.ceph.com' reimaging 2023-12-19T13:09:54.888 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: reset os type and version on server 2023-12-19T13:09:54.888 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2023-12-19T13:09:54.907 INFO:teuthology.lock.ops:Start node 'smithi105.front.sepia.ceph.com' reimaging 2023-12-19T13:09:54.907 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: reset os type and version on server 2023-12-19T13:09:54.907 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-12-19T13:09:55.020 DEBUG:teuthology.provision.fog.smithi029:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-19T13:09:55.032 DEBUG:teuthology.provision.fog.smithi105:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-19T13:09:55.063 INFO:teuthology.provision.fog.smithi029:Scheduling deploy of rhel 8.6 2023-12-19T13:09:55.085 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of rhel 8.6 2023-12-19T13:09:55.375 INFO:teuthology.orchestra.console.smithi105:Power off 2023-12-19T13:09:55.375 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T13:09:55.389 INFO:teuthology.orchestra.console.smithi029:Power off 2023-12-19T13:09:55.389 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T13:09:55.397 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2023-12-19T13:09:55.397 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:09:55.411 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2023-12-19T13:09:55.412 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:09:55.420 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2023-12-19T13:09:55.433 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-12-19T13:09:55.521 INFO:teuthology.orchestra.console.smithi105:Power off completed 2023-12-19T13:09:55.534 INFO:teuthology.orchestra.console.smithi029:Power off completed 2023-12-19T13:09:55.622 INFO:teuthology.orchestra.console.smithi105:Power on 2023-12-19T13:09:55.622 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-19T13:09:55.634 INFO:teuthology.orchestra.console.smithi029:Power on 2023-12-19T13:09:55.635 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-19T13:09:55.645 DEBUG:teuthology.orchestra.console.smithi105:power on output: Chassis Power Control: Up/On 2023-12-19T13:09:55.645 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:09:55.655 DEBUG:teuthology.orchestra.console.smithi029:power on output: Chassis Power Control: Up/On 2023-12-19T13:09:55.655 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:09:55.665 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2023-12-19T13:09:55.676 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-12-19T13:09:59.669 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:09:59.676 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:09:59.778 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2023-12-19T13:09:59.783 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-12-19T13:09:59.880 INFO:teuthology.orchestra.console.smithi105:Power on completed 2023-12-19T13:09:59.980 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish 2023-12-19T13:10:03.787 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:10:03.895 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-12-19T13:10:03.996 INFO:teuthology.orchestra.console.smithi029:Power on completed 2023-12-19T13:10:04.097 INFO:teuthology.provision.fog.smithi029:Waiting for deploy to finish 2023-12-19T13:12:46.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:12:49.233 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-12-19T13:12:55.238 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:12:58.289 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-12-19T13:13:04.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:07.345 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-12-19T13:13:13.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:16.401 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-12-19T13:13:20.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:22.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:23.381 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-12-19T13:13:25.461 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-12-19T13:13:29.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:31.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:32.465 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-12-19T13:13:34.545 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-12-19T13:13:38.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:40.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:41.525 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-12-19T13:13:47.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:50.613 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-12-19T13:13:56.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:13:59.697 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-12-19T13:14:05.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:14:08.753 WARNING:teuthology.provision.fog.smithi029:[Errno None] Unable to connect to port 22 on 172.21.15.29 2023-12-19T13:14:14.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:14:40.574 WARNING:teuthology.provision.fog.smithi105:timed out 2023-12-19T13:14:46.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:14:46.789 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-19T13:14:46.790 DEBUG:teuthology.orchestra.run.smithi105:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-19T13:14:47.053 INFO:teuthology.provision.fog.smithi105:Node is ready 2023-12-19T13:14:47.053 DEBUG:teuthology.orchestra.run.smithi105:> hostname 2023-12-19T13:14:47.071 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105 2023-12-19T13:14:47.072 DEBUG:teuthology.orchestra.run.smithi105:> grep smithi105 /etc/hosts 2023-12-19T13:14:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105 smithi105.front.sepia.ceph.com smithi105 2023-12-19T13:14:47.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname smithi105 2023-12-19T13:14:47.250 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/smithi105/smithi105/g' /etc/hosts 2023-12-19T13:14:47.279 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/172.21.15.105/172.21.15.105/g' /etc/hosts 2023-12-19T13:14:47.348 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/smithi105/smithi105/g' /etc/hostname 2023-12-19T13:14:47.415 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname smithi105 2023-12-19T13:14:47.483 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2023-12-19T13:14:47.540 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T13:14:47.540 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8.6 (Ootpa)" 2023-12-19T13:14:47.540 INFO:teuthology.orchestra.run.smithi105.stdout:ID="rhel" 2023-12-19T13:14:47.540 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="fedora" 2023-12-19T13:14:47.540 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8.6" 2023-12-19T13:14:47.540 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2023-12-19T13:14:47.540 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T13:14:47.541 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-19T13:14:47.542 INFO:teuthology.provision.fog.smithi105:Deploy complete! 2023-12-19T13:14:47.543 DEBUG:teuthology.parallel:result is None 2023-12-19T13:15:14.786 WARNING:teuthology.provision.fog.smithi029:timed out 2023-12-19T13:15:20.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:15:21.043 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-19T13:15:21.044 DEBUG:teuthology.orchestra.run.smithi029:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-19T13:15:21.334 INFO:teuthology.provision.fog.smithi029:Node is ready 2023-12-19T13:15:21.334 DEBUG:teuthology.orchestra.run.smithi029:> hostname 2023-12-19T13:15:21.355 INFO:teuthology.orchestra.run.smithi029.stdout:smithi029.front.sepia.ceph.com 2023-12-19T13:15:21.355 DEBUG:teuthology.orchestra.run.smithi029:> grep smithi029.front.sepia.ceph.com /etc/hosts 2023-12-19T13:15:21.416 INFO:teuthology.orchestra.run.smithi029.stdout:172.21.15.29 smithi029.front.sepia.ceph.com smithi029 2023-12-19T13:15:21.417 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname smithi029 2023-12-19T13:15:21.541 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/smithi029.front.sepia.ceph.com/smithi029/g' /etc/hosts 2023-12-19T13:15:21.577 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/172.21.15.29/172.21.15.29/g' /etc/hosts 2023-12-19T13:15:21.652 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i -e 's/smithi029.front.sepia.ceph.com/smithi029/g' /etc/hostname 2023-12-19T13:15:21.727 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname smithi029 2023-12-19T13:15:21.803 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2023-12-19T13:15:21.864 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T13:15:21.864 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8.6 (Ootpa)" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:ID="rhel" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="fedora" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8.6" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T13:15:21.865 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-19T13:15:21.866 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T13:15:21.866 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-19T13:15:21.866 INFO:teuthology.provision.fog.smithi029:Deploy complete! 2023-12-19T13:15:21.867 DEBUG:teuthology.parallel:result is None 2023-12-19T13:15:22.019 INFO:teuthology.lock.ops:Checking smithi029.front.sepia.ceph.com 2023-12-19T13:15:22.019 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-19T13:15:22.047 INFO:teuthology.lock.ops:Checking smithi105.front.sepia.ceph.com 2023-12-19T13:15:22.047 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-19T13:15:22.064 INFO:teuthology.lock.ops:Updating [smithi029.front.sepia.ceph.com]: set os type and version on server 2023-12-19T13:15:22.065 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2023-12-19T13:15:22.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:15:22.134 DEBUG:teuthology.orchestra.run.smithi029:> true 2023-12-19T13:15:22.206 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2023-12-19T13:15:22.207 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2023-12-19T13:15:22.268 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2023-12-19T13:15:22.268 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2023-12-19T13:15:22.329 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T13:15:22.329 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8.6 (Ootpa)" 2023-12-19T13:15:22.329 INFO:teuthology.orchestra.run.smithi029.stdout:ID="rhel" 2023-12-19T13:15:22.329 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="fedora" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8.6" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T13:15:22.330 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-19T13:15:22.331 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2023-12-19T13:15:22.355 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: set os type and version on server 2023-12-19T13:15:22.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-12-19T13:15:22.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T13:15:22.422 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-12-19T13:15:22.558 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-12-19T13:15:22.559 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2023-12-19T13:15:22.616 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2023-12-19T13:15:22.616 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Red Hat Enterprise Linux" 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8.6 (Ootpa)" 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:ID="rhel" 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="fedora" 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8.6" 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2023-12-19T13:15:22.673 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-19T13:15:22.674 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.redhat.com/" 2023-12-19T13:15:22.674 INFO:teuthology.orchestra.run.smithi105.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-19T13:15:22.674 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T13:15:22.674 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-12-19T13:15:22.674 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-19T13:15:22.674 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-19T13:15:22.674 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-19T13:15:22.674 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-19T13:15:22.674 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-12-19T13:15:22.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T13:15:22.792 INFO:teuthology.dispatcher.supervisor:Running job 7496740 2023-12-19T13:15:22.792 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-12-19_11:42:11-fs-rishabh-mdscap-reef-testing-default-smithi/7496740 --name rishabh-2023-12-19_11:42:11-fs-rishabh-mdscap-reef-testing-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_fsstress}} -- /home/teuthworker/archive/rishabh-2023-12-19_11:42:11-fs-rishabh-mdscap-reef-testing-default-smithi/7496740/orig.config.yaml 2023-12-19T13:15:22.795 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2023-12-19_11:42:11-fs-rishabh-mdscap-reef-testing-default-smithi/7496740 2023-12-19T13:15:22.795 INFO:teuthology.dispatcher.supervisor:Job PID: 996991 2023-12-19T13:15:22.795 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-19T13:17:22.894 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T13:19:23.066 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T13:21:23.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T13:23:23.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T13:25:23.539 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T13:25:23.606 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-19T13:25:23.734 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-19T13:25:23.735 INFO:teuthology.nuke:Checking targets against current locks 2023-12-19T13:25:23.782 DEBUG:teuthology.nuke:shortname: smithi029 2023-12-19T13:25:23.782 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T13:25:23.807 DEBUG:teuthology.nuke:shortname: smithi105 2023-12-19T13:25:23.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T13:25:23.811 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_11:42:11-fs-rishabh-mdscap-reef-testing-default-smithi/7496740', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 13:09:53.727790', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOfYiIr0z07cfrl4GNScBBcreb1f7yyJRONFaHPyTVMUQ8UmJFpI1uNvTkTXAcqNEq0oamyNmU48oGRUfJzs6Bg='} 2023-12-19T13:25:23.829 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-12-19_11:42:11-fs-rishabh-mdscap-reef-testing-default-smithi/7496740', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 13:09:53.726620', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFFVxfisCbVoFcfrclvtHHZl6ZxrwvjbF2+24401vWR9urTwVlDM1IPD/LyJpaSY+wnhQacBhdVXKfjwTleVsbs='} 2023-12-19T13:25:23.861 INFO:teuthology.orchestra.console.smithi029:Power off 2023-12-19T13:25:23.861 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T13:25:23.888 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2023-12-19T13:25:23.889 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:25:23.904 INFO:teuthology.orchestra.console.smithi105:Power off 2023-12-19T13:25:23.905 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T13:25:23.914 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-12-19T13:25:23.926 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2023-12-19T13:25:23.927 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T13:25:23.949 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2023-12-19T13:25:24.015 INFO:teuthology.orchestra.console.smithi029:Power off completed 2023-12-19T13:25:24.051 INFO:teuthology.orchestra.console.smithi105:Power off completed 2023-12-19T13:25:24.172 INFO:teuthology.lock.ops:unlocked: smithi029.front.sepia.ceph.com 2023-12-19T13:25:24.197 INFO:teuthology.lock.ops:unlocked: smithi105.front.sepia.ceph.com