2023-12-20T03:59:51.679 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-20T03:59:51.679 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T03:59:52.081 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbc6e3b1940>, signals=[15]) 2023-12-20T03:59:52.082 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging 2023-12-20T03:59:52.082 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server 2023-12-20T03:59:52.082 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-12-20T03:59:52.096 INFO:teuthology.lock.ops:Start node 'smithi190.front.sepia.ceph.com' reimaging 2023-12-20T03:59:52.097 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: reset os type and version on server 2023-12-20T03:59:52.097 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-12-20T03:59:52.219 DEBUG:teuthology.provision.fog.smithi190:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-20T03:59:52.230 DEBUG:teuthology.provision.fog.smithi154:Requesting image smithi_rhel_8.6 (ID 5) 2023-12-20T03:59:52.266 INFO:teuthology.provision.fog.smithi190:Scheduling deploy of rhel 8.6 2023-12-20T03:59:52.282 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of rhel 8.6 2023-12-20T03:59:52.750 INFO:teuthology.orchestra.console.smithi190:Power off 2023-12-20T03:59:52.751 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-20T03:59:52.775 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-12-20T03:59:52.775 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T03:59:52.794 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-12-20T03:59:52.801 INFO:teuthology.orchestra.console.smithi154:Power off 2023-12-20T03:59:52.802 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-20T03:59:52.821 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2023-12-20T03:59:52.822 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T03:59:52.842 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-12-20T03:59:52.896 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-12-20T03:59:52.943 INFO:teuthology.orchestra.console.smithi154:Power off completed 2023-12-20T03:59:52.997 INFO:teuthology.orchestra.console.smithi190:Power on 2023-12-20T03:59:52.997 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-20T03:59:53.017 DEBUG:teuthology.orchestra.console.smithi190:power on output: Chassis Power Control: Up/On 2023-12-20T03:59:53.017 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T03:59:53.038 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-12-20T03:59:53.044 INFO:teuthology.orchestra.console.smithi154:Power on 2023-12-20T03:59:53.044 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-20T03:59:53.065 DEBUG:teuthology.orchestra.console.smithi154:power on output: Chassis Power Control: Up/On 2023-12-20T03:59:53.065 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T03:59:53.087 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-12-20T03:59:57.043 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T03:59:57.087 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T03:59:57.155 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-12-20T03:59:57.197 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-12-20T04:00:01.160 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T04:00:01.198 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T04:00:01.268 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-12-20T04:00:01.305 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-12-20T04:00:01.370 INFO:teuthology.orchestra.console.smithi190:Power on completed 2023-12-20T04:00:01.406 INFO:teuthology.orchestra.console.smithi154:Power on completed 2023-12-20T04:00:01.470 INFO:teuthology.provision.fog.smithi190:Waiting for deploy to finish 2023-12-20T04:00:01.507 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish 2023-12-20T04:05:35.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:05:38.770 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:05:44.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:05:47.857 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:05:53.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:05:56.914 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:06:02.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:05.969 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:06:11.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:15.058 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:06:21.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:21.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:24.114 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:06:24.213 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:06:30.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:30.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:33.265 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:06:39.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:42.357 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:06:48.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:48.593 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:06:51.446 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:06:54.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:57.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:06:57.649 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:07:00.529 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:07:03.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:07:06.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:07:06.705 WARNING:teuthology.provision.fog.smithi190:[Errno None] Unable to connect to port 22 on 172.21.15.190 2023-12-20T04:07:09.585 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:07:12.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:07:13.651 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-20T04:07:13.651 DEBUG:teuthology.orchestra.run.smithi190:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-20T04:07:15.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:07:28.909 INFO:teuthology.provision.fog.smithi190:Node is ready 2023-12-20T04:07:28.910 DEBUG:teuthology.orchestra.run.smithi190:> hostname 2023-12-20T04:07:28.972 INFO:teuthology.orchestra.run.smithi190.stdout:smithi190.front.sepia.ceph.com 2023-12-20T04:07:28.972 DEBUG:teuthology.orchestra.run.smithi190:> grep smithi190.front.sepia.ceph.com /etc/hosts 2023-12-20T04:07:29.031 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.15.190 smithi190.front.sepia.ceph.com smithi190 2023-12-20T04:07:29.031 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname smithi190 2023-12-20T04:07:29.152 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/smithi190.front.sepia.ceph.com/smithi190/g' /etc/hosts 2023-12-20T04:07:29.181 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/172.21.15.190/172.21.15.190/g' /etc/hosts 2023-12-20T04:07:29.250 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i -e 's/smithi190.front.sepia.ceph.com/smithi190/g' /etc/hostname 2023-12-20T04:07:29.318 DEBUG:teuthology.orchestra.run.smithi190:> sudo hostname smithi190 2023-12-20T04:07:29.385 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Red Hat Enterprise Linux" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8.6 (Ootpa)" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:ID="rhel" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="fedora" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8.6" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-20T04:07:29.442 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.redhat.com/" 2023-12-20T04:07:29.443 INFO:teuthology.orchestra.run.smithi190.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-20T04:07:29.443 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-20T04:07:29.443 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-20T04:07:29.443 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-20T04:07:29.443 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-20T04:07:29.443 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-20T04:07:29.443 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-20T04:07:29.444 INFO:teuthology.provision.fog.smithi190:Deploy complete! 2023-12-20T04:07:29.445 DEBUG:teuthology.parallel:result is None 2023-12-20T04:07:33.905 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:07:39.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:07:42.961 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:07:48.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:07:52.017 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-12-20T04:07:58.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:07:58.272 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-20T04:07:58.273 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-20T04:08:18.541 INFO:teuthology.provision.fog.smithi154:Node is ready 2023-12-20T04:08:18.541 DEBUG:teuthology.orchestra.run.smithi154:> hostname 2023-12-20T04:08:18.562 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com 2023-12-20T04:08:18.563 DEBUG:teuthology.orchestra.run.smithi154:> grep smithi154.front.sepia.ceph.com /etc/hosts 2023-12-20T04:08:18.622 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154 2023-12-20T04:08:18.623 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154 2023-12-20T04:08:18.750 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hosts 2023-12-20T04:08:18.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/172.21.15.154/172.21.15.154/g' /etc/hosts 2023-12-20T04:08:18.850 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hostname 2023-12-20T04:08:18.918 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154 2023-12-20T04:08:18.987 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-12-20T04:08:19.045 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Red Hat Enterprise Linux" 2023-12-20T04:08:19.045 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8.6 (Ootpa)" 2023-12-20T04:08:19.045 INFO:teuthology.orchestra.run.smithi154.stdout:ID="rhel" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="fedora" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8.6" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.redhat.com/" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-20T04:08:19.046 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-20T04:08:19.047 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-12-20T04:08:19.047 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-20T04:08:19.047 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-20T04:08:19.047 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-20T04:08:19.047 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-20T04:08:19.047 INFO:teuthology.provision.fog.smithi154:Deploy complete! 2023-12-20T04:08:19.048 DEBUG:teuthology.parallel:result is None 2023-12-20T04:08:19.199 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com 2023-12-20T04:08:19.199 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-20T04:08:19.225 INFO:teuthology.lock.ops:Checking smithi190.front.sepia.ceph.com 2023-12-20T04:08:19.225 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-20T04:08:19.243 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server 2023-12-20T04:08:19.243 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-12-20T04:08:19.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:08:19.311 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-12-20T04:08:19.378 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-12-20T04:08:19.378 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2023-12-20T04:08:19.435 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2023-12-20T04:08:19.435 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-12-20T04:08:19.491 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="Red Hat Enterprise Linux" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="8.6 (Ootpa)" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:ID="rhel" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="fedora" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="8.6" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el8" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://www.redhat.com/" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-12-20T04:08:19.492 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-20T04:08:19.493 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-20T04:08:19.493 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-20T04:08:19.493 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-20T04:08:19.493 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-12-20T04:08:19.534 INFO:teuthology.lock.ops:Updating [smithi190.front.sepia.ceph.com]: set os type and version on server 2023-12-20T04:08:19.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-12-20T04:08:19.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:08:19.601 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-12-20T04:08:19.743 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-12-20T04:08:19.743 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2023-12-20T04:08:19.800 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2023-12-20T04:08:19.801 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-12-20T04:08:19.858 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="Red Hat Enterprise Linux" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="8.6 (Ootpa)" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:ID="rhel" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="fedora" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="8.6" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el8" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://www.redhat.com/" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-20T04:08:19.859 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-20T04:08:19.860 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-12-20T04:08:19.860 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-20T04:08:19.860 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-20T04:08:19.860 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-20T04:08:19.860 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-20T04:08:19.860 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-12-20T04:08:19.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T04:08:19.991 INFO:teuthology.dispatcher.supervisor:Running job 7497403 2023-12-20T04:08:19.991 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497403 --name vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_20.04} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/no races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/mds 2-workunit/suites/pjd}} -- /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497403/orig.config.yaml 2023-12-20T04:08:19.995 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497403 2023-12-20T04:08:19.995 INFO:teuthology.dispatcher.supervisor:Job PID: 3420429 2023-12-20T04:08:19.995 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-20T04:10:20.094 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T04:12:20.262 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T04:14:20.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T04:16:20.582 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T04:18:20.750 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T04:18:20.951 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-20T04:18:21.080 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-20T04:18:21.080 INFO:teuthology.nuke:Checking targets against current locks 2023-12-20T04:18:21.129 DEBUG:teuthology.nuke:shortname: smithi154 2023-12-20T04:18:21.129 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-20T04:18:21.151 DEBUG:teuthology.nuke:shortname: smithi190 2023-12-20T04:18:21.151 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-20T04:18:21.155 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497403', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-20 03:59:50.847006', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBNWQ5RkS+SGsk+01h78IngOoMzw8WkTe4PWhYMYGMHht100hgXSzQSQu0PAh0DaZPNA0MnHq/irFeXdPJ2VkSM='} 2023-12-20T04:18:21.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497403', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-20 03:59:50.848265', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9Yi5YArJrwaar5FmCAwUUch//8PZKSjxVjeLbs50sN4c4bIYEj18MUF3QJnJsDvRJ5gHtFQQxlEWWOjQTl99s='} 2023-12-20T04:18:21.213 INFO:teuthology.orchestra.console.smithi154:Power off 2023-12-20T04:18:21.213 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-20T04:18:21.234 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2023-12-20T04:18:21.234 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T04:18:21.250 INFO:teuthology.orchestra.console.smithi190:Power off 2023-12-20T04:18:21.250 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-20T04:18:21.260 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-12-20T04:18:21.273 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-12-20T04:18:21.273 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T04:18:21.301 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-12-20T04:18:21.361 INFO:teuthology.orchestra.console.smithi154:Power off completed 2023-12-20T04:18:21.402 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-12-20T04:18:21.518 INFO:teuthology.lock.ops:unlocked: smithi154.front.sepia.ceph.com 2023-12-20T04:18:21.550 INFO:teuthology.lock.ops:unlocked: smithi190.front.sepia.ceph.com