2024-01-08T06:52:27.974 INFO:root:teuthology version: 0.0.1.dev228+g01dd171 2024-01-08T06:52:27.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T06:52:28.094 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc2760fd940>, signals=[15]) 2024-01-08T06:52:28.094 INFO:teuthology.lock.ops:Start node 'smithi084.front.sepia.ceph.com' reimaging 2024-01-08T06:52:28.095 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: reset os type and version on server 2024-01-08T06:52:28.095 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-01-08T06:52:28.114 INFO:teuthology.lock.ops:Start node 'smithi161.front.sepia.ceph.com' reimaging 2024-01-08T06:52:28.114 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: reset os type and version on server 2024-01-08T06:52:28.115 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-01-08T06:52:28.240 DEBUG:teuthology.provision.fog.smithi084:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-08T06:52:28.274 DEBUG:teuthology.provision.fog.smithi161:Requesting image smithi_rhel_8.6 (ID 5) 2024-01-08T06:52:28.307 INFO:teuthology.provision.fog.smithi084:Scheduling deploy of rhel 8.6 2024-01-08T06:52:28.348 INFO:teuthology.provision.fog.smithi161:Scheduling deploy of rhel 8.6 2024-01-08T06:52:29.051 INFO:teuthology.orchestra.console.smithi161:Power off 2024-01-08T06:52:29.051 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-08T06:52:29.072 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2024-01-08T06:52:29.072 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:29.096 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-01-08T06:52:29.205 INFO:teuthology.orchestra.console.smithi084:Power off 2024-01-08T06:52:29.206 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-08T06:52:29.226 DEBUG:teuthology.orchestra.console.smithi084:power off output: Chassis Power Control: Down/Off 2024-01-08T06:52:29.227 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:29.253 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-01-08T06:52:33.100 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:33.208 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-01-08T06:52:33.254 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:33.361 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-01-08T06:52:37.213 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:37.321 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-01-08T06:52:37.362 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:37.469 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-01-08T06:52:41.326 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:41.433 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-01-08T06:52:41.471 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:41.534 INFO:teuthology.orchestra.console.smithi161:Power off completed 2024-01-08T06:52:41.579 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2024-01-08T06:52:41.636 INFO:teuthology.orchestra.console.smithi161:Power on 2024-01-08T06:52:41.636 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-08T06:52:41.656 DEBUG:teuthology.orchestra.console.smithi161:power on output: Chassis Power Control: Up/On 2024-01-08T06:52:41.656 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:41.676 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-01-08T06:52:41.680 INFO:teuthology.orchestra.console.smithi084:Power off completed 2024-01-08T06:52:41.781 INFO:teuthology.orchestra.console.smithi084:Power on 2024-01-08T06:52:41.781 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-01-08T06:52:41.801 DEBUG:teuthology.orchestra.console.smithi084:power on output: Chassis Power Control: Up/On 2024-01-08T06:52:41.801 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:41.821 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2024-01-08T06:52:45.680 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:45.787 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-01-08T06:52:45.822 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:45.929 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2024-01-08T06:52:49.788 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:49.898 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-01-08T06:52:49.930 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T06:52:49.999 INFO:teuthology.orchestra.console.smithi161:Power on completed 2024-01-08T06:52:50.038 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is on 2024-01-08T06:52:50.101 INFO:teuthology.provision.fog.smithi161:Waiting for deploy to finish 2024-01-08T06:52:50.139 INFO:teuthology.orchestra.console.smithi084:Power on completed 2024-01-08T06:52:50.240 INFO:teuthology.provision.fog.smithi084:Waiting for deploy to finish 2024-01-08T06:59:30.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T06:59:33.628 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T06:59:39.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T06:59:42.684 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T06:59:48.688 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T06:59:51.745 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T06:59:57.748 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:00.832 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T07:00:06.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:09.916 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T07:00:15.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:15.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:18.844 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-01-08T07:00:18.972 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T07:00:24.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:24.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:27.900 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-01-08T07:00:28.032 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T07:00:33.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:34.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:36.956 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-01-08T07:00:42.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:46.048 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-01-08T07:00:52.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:00:52.412 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T07:00:55.132 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-01-08T07:00:58.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:01:01.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:01:01.472 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T07:01:04.192 WARNING:teuthology.provision.fog.smithi084:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-01-08T07:01:07.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:01:10.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:01:10.556 WARNING:teuthology.provision.fog.smithi161:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-01-08T07:01:16.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:01:16.773 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-08T07:01:16.773 DEBUG:teuthology.orchestra.run.smithi161:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-08T07:01:37.026 INFO:teuthology.provision.fog.smithi161:Node is ready 2024-01-08T07:01:37.026 DEBUG:teuthology.orchestra.run.smithi161:> hostname 2024-01-08T07:01:37.049 INFO:teuthology.orchestra.run.smithi161.stdout:smithi161.front.sepia.ceph.com 2024-01-08T07:01:37.049 DEBUG:teuthology.orchestra.run.smithi161:> grep smithi161.front.sepia.ceph.com /etc/hosts 2024-01-08T07:01:37.108 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.15.161 smithi161.front.sepia.ceph.com smithi161 2024-01-08T07:01:37.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161 2024-01-08T07:01:37.243 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hosts 2024-01-08T07:01:37.274 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/172.21.15.161/172.21.15.161/g' /etc/hosts 2024-01-08T07:01:37.343 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i -e 's/smithi161.front.sepia.ceph.com/smithi161/g' /etc/hostname 2024-01-08T07:01:37.413 DEBUG:teuthology.orchestra.run.smithi161:> sudo hostname smithi161 2024-01-08T07:01:37.480 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-08T07:01:37.537 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-08T07:01:37.538 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-01-08T07:01:37.538 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-08T07:01:37.538 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-08T07:01:37.538 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-08T07:01:37.538 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-08T07:01:37.538 INFO:teuthology.provision.fog.smithi161:Deploy complete! 2024-01-08T07:01:37.539 DEBUG:teuthology.parallel:result is None 2024-01-08T07:02:10.227 WARNING:teuthology.provision.fog.smithi084:timed out 2024-01-08T07:02:16.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:02:16.463 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-01-08T07:02:16.463 DEBUG:teuthology.orchestra.run.smithi084:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-01-08T07:02:16.755 INFO:teuthology.provision.fog.smithi084:Node is ready 2024-01-08T07:02:16.755 DEBUG:teuthology.orchestra.run.smithi084:> hostname 2024-01-08T07:02:16.774 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084.front.sepia.ceph.com 2024-01-08T07:02:16.774 DEBUG:teuthology.orchestra.run.smithi084:> grep smithi084.front.sepia.ceph.com /etc/hosts 2024-01-08T07:02:16.832 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84 smithi084.front.sepia.ceph.com smithi084 2024-01-08T07:02:16.832 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname smithi084 2024-01-08T07:02:16.952 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/smithi084.front.sepia.ceph.com/smithi084/g' /etc/hosts 2024-01-08T07:02:16.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/172.21.15.84/172.21.15.84/g' /etc/hosts 2024-01-08T07:02:17.049 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i -e 's/smithi084.front.sepia.ceph.com/smithi084/g' /etc/hostname 2024-01-08T07:02:17.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname smithi084 2024-01-08T07:02:17.185 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-01-08T07:02:17.242 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Red Hat Enterprise Linux" 2024-01-08T07:02:17.242 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8.6 (Ootpa)" 2024-01-08T07:02:17.242 INFO:teuthology.orchestra.run.smithi084.stdout:ID="rhel" 2024-01-08T07:02:17.242 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="fedora" 2024-01-08T07:02:17.242 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8.6" 2024-01-08T07:02:17.242 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8" 2024-01-08T07:02:17.242 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.redhat.com/" 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-08T07:02:17.243 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-08T07:02:17.243 INFO:teuthology.provision.fog.smithi084:Deploy complete! 2024-01-08T07:02:17.244 DEBUG:teuthology.parallel:result is None 2024-01-08T07:02:17.391 INFO:teuthology.lock.ops:Checking smithi084.front.sepia.ceph.com 2024-01-08T07:02:17.391 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-08T07:02:17.417 INFO:teuthology.lock.ops:Checking smithi161.front.sepia.ceph.com 2024-01-08T07:02:17.417 INFO:teuthology.lock.ops:New key found. Updating... 2024-01-08T07:02:17.434 INFO:teuthology.lock.ops:Updating [smithi084.front.sepia.ceph.com]: set os type and version on server 2024-01-08T07:02:17.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-01-08T07:02:17.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:02:17.500 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-01-08T07:02:17.524 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-01-08T07:02:17.524 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-01-08T07:02:17.581 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-01-08T07:02:17.581 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-01-08T07:02:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Red Hat Enterprise Linux" 2024-01-08T07:02:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8.6 (Ootpa)" 2024-01-08T07:02:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:ID="rhel" 2024-01-08T07:02:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="fedora" 2024-01-08T07:02:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8.6" 2024-01-08T07:02:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8" 2024-01-08T07:02:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-08T07:02:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.redhat.com/" 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-08T07:02:17.638 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-08T07:02:17.639 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-01-08T07:02:17.658 INFO:teuthology.lock.ops:Updating [smithi161.front.sepia.ceph.com]: set os type and version on server 2024-01-08T07:02:17.659 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-01-08T07:02:17.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-08T07:02:17.726 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-01-08T07:02:17.869 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-01-08T07:02:17.870 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-01-08T07:02:17.927 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-01-08T07:02:17.928 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-01-08T07:02:17.984 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Red Hat Enterprise Linux" 2024-01-08T07:02:17.984 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8.6 (Ootpa)" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:ID="rhel" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="fedora" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8.6" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.redhat.com/" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-01-08T07:02:17.985 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-01-08T07:02:17.986 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-01-08T07:02:17.986 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-01-08T07:02:18.008 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:02:18.090 INFO:teuthology.dispatcher.supervisor:Running job 7508885 2024-01-08T07:02:18.090 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_01dd171a561ca33631bf0a38b105879779f72410/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508885 --name vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi --description fs/multifs/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-2c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore-ec/bluestore-comp overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug} tasks/multifs-auth} -- /home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508885/orig.config.yaml 2024-01-08T07:02:18.093 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508885 2024-01-08T07:02:18.094 INFO:teuthology.dispatcher.supervisor:Job PID: 3657414 2024-01-08T07:02:18.094 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-01-08T07:04:18.193 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:06:18.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:08:18.525 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:10:18.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:12:18.854 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:14:19.025 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-08T07:14:19.097 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2024-01-08T07:14:19.237 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2024-01-08T07:14:19.237 INFO:teuthology.nuke:Checking targets against current locks 2024-01-08T07:14:19.288 DEBUG:teuthology.nuke:shortname: smithi084 2024-01-08T07:14:19.289 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-08T07:14:19.313 DEBUG:teuthology.nuke:shortname: smithi161 2024-01-08T07:14:19.313 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-08T07:14:19.320 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508885', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-08 06:52:27.195027', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIZyh2ztuO38Ng1joYUH7lTajzx2kRRRU8FuZrnJs9OFcetOazjbvEz4xhGNBnOPn+k8MeeIt1011ZQw76vOlnc='} 2024-01-08T07:14:19.349 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-08_06:50:55-fs-wip-vshankar-testing-20240103.072409-testing-default-smithi/7508885', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-08 06:52:27.196308', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO9/mk+wST+uMeoDynA4EPrZh7dbWZAjxgJkVf1q1uI8tYeOQhujAD3AidkL15NMJt6IyaN5eee6ZHrcr8qEYSI='} 2024-01-08T07:14:19.401 INFO:teuthology.orchestra.console.smithi084:Power off 2024-01-08T07:14:19.402 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-08T07:14:19.484 INFO:teuthology.orchestra.console.smithi161:Power off 2024-01-08T07:14:19.485 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-08T07:14:19.512 DEBUG:teuthology.orchestra.console.smithi084:power off output: Chassis Power Control: Down/Off 2024-01-08T07:14:19.513 DEBUG:teuthology.orchestra.console.smithi084:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:14:19.537 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2024-01-08T07:14:19.538 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-08T07:14:19.559 DEBUG:teuthology.orchestra.console.smithi084:check power output: Chassis Power is off 2024-01-08T07:14:19.561 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-01-08T07:14:19.661 INFO:teuthology.orchestra.console.smithi084:Power off completed 2024-01-08T07:14:19.662 INFO:teuthology.orchestra.console.smithi161:Power off completed 2024-01-08T07:14:19.819 INFO:teuthology.lock.ops:unlocked: smithi084.front.sepia.ceph.com 2024-01-08T07:14:19.829 INFO:teuthology.lock.ops:unlocked: smithi161.front.sepia.ceph.com