2023-12-16T04:33:48.948 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-16T04:33:48.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-16T04:33:49.093 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f258f317940>, signals=[15]) 2023-12-16T04:33:49.093 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging 2023-12-16T04:33:49.094 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server 2023-12-16T04:33:49.094 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-12-16T04:33:49.120 INFO:teuthology.lock.ops:Start node 'smithi149.front.sepia.ceph.com' reimaging 2023-12-16T04:33:49.120 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: reset os type and version on server 2023-12-16T04:33:49.121 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2023-12-16T04:33:49.241 DEBUG:teuthology.provision.fog.smithi016:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-16T04:33:49.253 DEBUG:teuthology.provision.fog.smithi149:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-16T04:33:49.290 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of rhel 8.6 2023-12-16T04:33:49.293 INFO:teuthology.provision.fog.smithi149:Scheduling deploy of rhel 8.6 2023-12-16T04:33:49.535 INFO:teuthology.orchestra.console.smithi016:Power off 2023-12-16T04:33:49.536 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-16T04:33:49.550 INFO:teuthology.orchestra.console.smithi149:Power off 2023-12-16T04:33:49.551 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-16T04:33:49.561 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-12-16T04:33:49.561 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:33:49.574 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2023-12-16T04:33:49.574 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:33:49.586 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-12-16T04:33:49.599 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2023-12-16T04:33:53.589 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:33:53.600 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:33:53.699 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-12-16T04:33:53.710 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2023-12-16T04:33:57.704 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:33:57.710 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:33:57.812 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-12-16T04:33:57.819 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2023-12-16T04:33:57.914 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-12-16T04:33:57.920 INFO:teuthology.orchestra.console.smithi149:Power off completed 2023-12-16T04:33:58.014 INFO:teuthology.orchestra.console.smithi016:Power on 2023-12-16T04:33:58.015 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-16T04:33:58.024 INFO:teuthology.orchestra.console.smithi149:Power on 2023-12-16T04:33:58.024 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-16T04:33:58.040 DEBUG:teuthology.orchestra.console.smithi016:power on output: Chassis Power Control: Up/On 2023-12-16T04:33:58.040 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:33:58.051 DEBUG:teuthology.orchestra.console.smithi149:power on output: Chassis Power Control: Up/On 2023-12-16T04:33:58.052 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:33:58.065 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-12-16T04:33:58.074 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2023-12-16T04:34:02.069 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:34:02.076 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:34:02.179 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-12-16T04:34:02.186 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2023-12-16T04:34:06.181 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:34:06.188 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:34:06.290 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-12-16T04:34:06.295 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2023-12-16T04:34:06.392 INFO:teuthology.orchestra.console.smithi016:Power on completed 2023-12-16T04:34:06.396 INFO:teuthology.orchestra.console.smithi149:Power on completed 2023-12-16T04:34:06.493 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2023-12-16T04:34:06.497 INFO:teuthology.provision.fog.smithi149:Waiting for deploy to finish 2023-12-16T04:36:37.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:36:40.402 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-16T04:36:46.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:36:49.457 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-16T04:36:52.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:36:55.441 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2023-12-16T04:36:55.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:36:58.513 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-16T04:37:01.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:04.497 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2023-12-16T04:37:04.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:07.569 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-16T04:37:10.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:13.553 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2023-12-16T04:37:13.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:16.625 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-16T04:37:19.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:22.609 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2023-12-16T04:37:22.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:25.681 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-16T04:37:28.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:31.665 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2023-12-16T04:37:31.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:37.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:40.722 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2023-12-16T04:37:46.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:37:49.809 WARNING:teuthology.provision.fog.smithi149:[Errno None] Unable to connect to port 22 on 172.21.15.149 2023-12-16T04:37:55.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:38:31.718 WARNING:teuthology.provision.fog.smithi016:timed out 2023-12-16T04:38:37.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:38:38.026 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-16T04:38:38.027 DEBUG:teuthology.orchestra.run.smithi016:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-16T04:38:38.294 INFO:teuthology.provision.fog.smithi016:Node is ready 2023-12-16T04:38:38.294 DEBUG:teuthology.orchestra.run.smithi016:> hostname 2023-12-16T04:38:38.314 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com 2023-12-16T04:38:38.314 DEBUG:teuthology.orchestra.run.smithi016:> grep smithi016.front.sepia.ceph.com /etc/hosts 2023-12-16T04:38:38.375 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016 2023-12-16T04:38:38.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2023-12-16T04:38:38.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hosts 2023-12-16T04:38:38.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/172.21.15.16/172.21.15.16/g' /etc/hosts 2023-12-16T04:38:38.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hostname 2023-12-16T04:38:38.682 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2023-12-16T04:38:38.756 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-12-16T04:38:38.817 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-16T04:38:38.818 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-16T04:38:38.819 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-16T04:38:38.819 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2023-12-16T04:38:38.820 DEBUG:teuthology.parallel:result is None 2023-12-16T04:38:55.833 WARNING:teuthology.provision.fog.smithi149:timed out 2023-12-16T04:39:01.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:39:02.073 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-16T04:39:02.073 DEBUG:teuthology.orchestra.run.smithi149:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-16T04:39:02.339 INFO:teuthology.provision.fog.smithi149:Node is ready 2023-12-16T04:39:02.340 DEBUG:teuthology.orchestra.run.smithi149:> hostname 2023-12-16T04:39:02.357 INFO:teuthology.orchestra.run.smithi149.stdout:smithi149 2023-12-16T04:39:02.358 DEBUG:teuthology.orchestra.run.smithi149:> grep smithi149 /etc/hosts 2023-12-16T04:39:02.416 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.15.149 smithi149.front.sepia.ceph.com smithi149 2023-12-16T04:39:02.417 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149 2023-12-16T04:39:02.528 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149/smithi149/g' /etc/hosts 2023-12-16T04:39:03.062 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/172.21.15.149/172.21.15.149/g' /etc/hosts 2023-12-16T04:39:03.138 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i -e 's/smithi149/smithi149/g' /etc/hostname 2023-12-16T04:39:03.211 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname smithi149 2023-12-16T04:39:03.281 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Red Hat Enterprise Linux" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8.6 (Ootpa)" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:ID="rhel" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="fedora" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8.6" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.redhat.com/" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-16T04:39:03.340 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-16T04:39:03.341 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-12-16T04:39:03.341 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-16T04:39:03.341 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-16T04:39:03.341 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-16T04:39:03.341 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-16T04:39:03.341 INFO:teuthology.provision.fog.smithi149:Deploy complete! 2023-12-16T04:39:03.342 DEBUG:teuthology.parallel:result is None 2023-12-16T04:39:03.551 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2023-12-16T04:39:03.552 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-16T04:39:03.575 INFO:teuthology.lock.ops:Checking smithi149.front.sepia.ceph.com 2023-12-16T04:39:03.575 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-16T04:39:03.601 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2023-12-16T04:39:03.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-16T04:39:03.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:39:03.677 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-12-16T04:39:03.821 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-16T04:39:03.821 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-12-16T04:39:03.882 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-12-16T04:39:03.882 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-12-16T04:39:03.943 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-12-16T04:39:03.943 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2023-12-16T04:39:03.943 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-12-16T04:39:03.943 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-12-16T04:39:03.943 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2023-12-16T04:39:03.943 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-12-16T04:39:03.943 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-16T04:39:03.944 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-16T04:39:03.945 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-12-16T04:39:03.967 INFO:teuthology.lock.ops:Updating [smithi149.front.sepia.ceph.com]: set os type and version on server 2023-12-16T04:39:03.967 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2023-12-16T04:39:03.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T04:39:04.035 DEBUG:teuthology.orchestra.run.smithi149:> true 2023-12-16T04:39:04.103 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2023-12-16T04:39:04.103 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2023-12-16T04:39:04.160 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2023-12-16T04:39:04.160 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Red Hat Enterprise Linux" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8.6 (Ootpa)" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:ID="rhel" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="fedora" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8.6" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-16T04:39:04.218 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.redhat.com/" 2023-12-16T04:39:04.219 INFO:teuthology.orchestra.run.smithi149.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-16T04:39:04.219 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-16T04:39:04.219 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-12-16T04:39:04.219 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-16T04:39:04.219 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-16T04:39:04.219 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-16T04:39:04.219 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-16T04:39:04.219 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2023-12-16T04:39:04.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-16T04:39:04.344 INFO:teuthology.dispatcher.supervisor:Running job 7493608 2023-12-16T04:39:04.344 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-16_00:09:48-fs-main-distro-default-smithi/7493608 --name yuriw-2023-12-16_00:09:48-fs-main-distro-default-smithi --description fs/thrash/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down multifs session_timeout thrashosds-health} tasks/{1-thrash/mon 2-workunit/cfuse_workunit_suites_pjd}} -- /home/teuthworker/archive/yuriw-2023-12-16_00:09:48-fs-main-distro-default-smithi/7493608/orig.config.yaml 2023-12-16T04:39:04.347 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-12-16_00:09:48-fs-main-distro-default-smithi/7493608 2023-12-16T04:39:04.348 INFO:teuthology.dispatcher.supervisor:Job PID: 492826 2023-12-16T04:39:04.348 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-16T04:41:04.446 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-16T04:43:04.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-16T04:45:04.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-16T04:47:04.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-16T04:47:04.994 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-16T04:47:05.106 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-16T04:47:05.107 INFO:teuthology.nuke:Checking targets against current locks 2023-12-16T04:47:05.157 DEBUG:teuthology.nuke:shortname: smithi016 2023-12-16T04:47:05.157 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-16T04:47:05.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-16_00:09:48-fs-main-distro-default-smithi/7493608', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-16 04:33:48.086284', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE5W9rv0ldAjvXhDPI18BH88H453l2suj3V3hf5hP5VzgHnEXXvX4W+R0j7tFMwugOnF/2/n2NEhBfncGgTknRo='} 2023-12-16T04:47:05.181 DEBUG:teuthology.nuke:shortname: smithi149 2023-12-16T04:47:05.181 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-16T04:47:05.205 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-16_00:09:48-fs-main-distro-default-smithi/7493608', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-16 04:33:48.084812', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOYqsC8UJpkvtvUp1L2iutC1QqP2xu/BrhxEj8edlRSOyxoa8uZ5C5mWhjVFVnaAyyp6knLJW6J0KqP+e2aznaI='} 2023-12-16T04:47:05.232 INFO:teuthology.orchestra.console.smithi016:Power off 2023-12-16T04:47:05.232 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-16T04:47:05.256 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-12-16T04:47:05.256 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:47:05.271 INFO:teuthology.orchestra.console.smithi149:Power off 2023-12-16T04:47:05.272 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-16T04:47:05.279 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-12-16T04:47:05.292 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2023-12-16T04:47:05.292 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T04:47:05.313 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2023-12-16T04:47:05.381 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-12-16T04:47:05.414 INFO:teuthology.orchestra.console.smithi149:Power off completed 2023-12-16T04:47:05.552 INFO:teuthology.lock.ops:unlocked: smithi016.front.sepia.ceph.com 2023-12-16T04:47:05.559 INFO:teuthology.lock.ops:unlocked: smithi149.front.sepia.ceph.com