2023-12-08T09:29:20.159 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-08T09:29:20.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:29:20.309 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb3ea0c1550>, signals=[15]) 2023-12-08T09:29:20.310 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging 2023-12-08T09:29:20.310 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server 2023-12-08T09:29:20.310 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-12-08T09:29:20.342 INFO:teuthology.lock.ops:Start node 'smithi151.front.sepia.ceph.com' reimaging 2023-12-08T09:29:20.343 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: reset os type and version on server 2023-12-08T09:29:20.343 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2023-12-08T09:29:20.458 DEBUG:teuthology.provision.fog.smithi146:Requesting image smithi_centos_8.stream (ID 3) 2023-12-08T09:29:20.479 DEBUG:teuthology.provision.fog.smithi151:Requesting image smithi_centos_8.stream (ID 3) 2023-12-08T09:29:20.503 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 8.stream 2023-12-08T09:29:20.532 INFO:teuthology.provision.fog.smithi151:Scheduling deploy of centos 8.stream 2023-12-08T09:29:20.731 INFO:teuthology.orchestra.console.smithi146:Power off 2023-12-08T09:29:20.732 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-08T09:29:20.763 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2023-12-08T09:29:20.763 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:20.782 INFO:teuthology.orchestra.console.smithi151:Power off 2023-12-08T09:29:20.782 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-08T09:29:20.791 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-12-08T09:29:20.804 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off 2023-12-08T09:29:20.804 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:20.828 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2023-12-08T09:29:24.794 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:24.829 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:24.904 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-12-08T09:29:24.938 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2023-12-08T09:29:28.906 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:28.939 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:29.016 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-12-08T09:29:29.048 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2023-12-08T09:29:29.118 INFO:teuthology.orchestra.console.smithi146:Power off completed 2023-12-08T09:29:29.149 INFO:teuthology.orchestra.console.smithi151:Power off completed 2023-12-08T09:29:29.219 INFO:teuthology.orchestra.console.smithi146:Power on 2023-12-08T09:29:29.220 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-08T09:29:29.243 DEBUG:teuthology.orchestra.console.smithi146:power on output: Chassis Power Control: Up/On 2023-12-08T09:29:29.243 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:29.252 INFO:teuthology.orchestra.console.smithi151:Power on 2023-12-08T09:29:29.252 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2023-12-08T09:29:29.268 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-12-08T09:29:29.275 DEBUG:teuthology.orchestra.console.smithi151:power on output: Chassis Power Control: Up/On 2023-12-08T09:29:29.276 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:29.301 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2023-12-08T09:29:33.270 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:33.302 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:33.383 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-12-08T09:29:33.412 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2023-12-08T09:29:37.386 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:37.413 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T09:29:37.497 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2023-12-08T09:29:37.523 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2023-12-08T09:29:37.598 INFO:teuthology.orchestra.console.smithi146:Power on completed 2023-12-08T09:29:37.624 INFO:teuthology.orchestra.console.smithi151:Power on completed 2023-12-08T09:29:37.701 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish 2023-12-08T09:29:37.725 INFO:teuthology.provision.fog.smithi151:Waiting for deploy to finish 2023-12-08T09:32:23.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:32:23.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:32:26.882 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:32:26.883 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-12-08T09:32:32.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:32:32.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:32:35.941 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:32:35.942 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-12-08T09:32:41.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:32:41.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:32:45.026 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:32:45.027 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-12-08T09:32:51.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:32:51.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:32:54.114 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-12-08T09:32:54.118 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:33:00.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:00.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:03.170 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-12-08T09:33:03.206 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:33:09.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:09.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:12.230 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-12-08T09:33:12.262 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:33:18.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:18.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:21.314 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:33:21.314 WARNING:teuthology.provision.fog.smithi151:[Errno None] Unable to connect to port 22 on 172.21.15.151 2023-12-08T09:33:27.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:27.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:30.402 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:33:36.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:33:54.914 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:34:00.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:34:03.970 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2023-12-08T09:34:09.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:34:12.634 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-08T09:34:12.635 DEBUG:teuthology.orchestra.run.smithi146:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-08T09:34:27.326 WARNING:teuthology.provision.fog.smithi151:timed out 2023-12-08T09:34:33.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:34:33.526 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-12-08T09:34:33.527 DEBUG:teuthology.orchestra.run.smithi151:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2023-12-08T09:34:33.778 INFO:teuthology.provision.fog.smithi151:Node is ready 2023-12-08T09:34:33.779 DEBUG:teuthology.orchestra.run.smithi151:> hostname 2023-12-08T09:34:33.798 INFO:teuthology.orchestra.run.smithi151.stdout:smithi151.front.sepia.ceph.com 2023-12-08T09:34:33.798 DEBUG:teuthology.orchestra.run.smithi151:> grep smithi151.front.sepia.ceph.com /etc/hosts 2023-12-08T09:34:33.858 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.15.151 smithi151.front.sepia.ceph.com smithi151 2023-12-08T09:34:33.858 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151 2023-12-08T09:34:33.963 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi151.front.sepia.ceph.com/smithi151/g' /etc/hosts 2023-12-08T09:34:33.993 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/172.21.15.151/172.21.15.151/g' /etc/hosts 2023-12-08T09:34:34.061 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i -e 's/smithi151.front.sepia.ceph.com/smithi151/g' /etc/hostname 2023-12-08T09:34:34.128 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname smithi151 2023-12-08T09:34:34.195 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8" 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8" 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8" 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2023-12-08T09:34:34.254 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-08T09:34:34.255 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2023-12-08T09:34:34.255 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T09:34:34.255 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-08T09:34:34.255 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T09:34:34.255 INFO:teuthology.provision.fog.smithi151:Deploy complete! 2023-12-08T09:34:34.257 DEBUG:teuthology.parallel:result is None 2023-12-08T09:34:38.018 INFO:teuthology.provision.fog.smithi146:Node is ready 2023-12-08T09:34:38.018 DEBUG:teuthology.orchestra.run.smithi146:> hostname 2023-12-08T09:34:38.043 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com 2023-12-08T09:34:38.044 DEBUG:teuthology.orchestra.run.smithi146:> grep smithi146.front.sepia.ceph.com /etc/hosts 2023-12-08T09:34:38.105 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146 2023-12-08T09:34:38.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146 2023-12-08T09:34:38.222 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hosts 2023-12-08T09:34:38.253 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/172.21.15.146/172.21.15.146/g' /etc/hosts 2023-12-08T09:34:38.323 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hostname 2023-12-08T09:34:38.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146 2023-12-08T09:34:38.459 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-12-08T09:34:38.517 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-08T09:34:38.518 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T09:34:38.519 INFO:teuthology.provision.fog.smithi146:Deploy complete! 2023-12-08T09:34:38.519 DEBUG:teuthology.parallel:result is None 2023-12-08T09:34:38.716 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com 2023-12-08T09:34:38.716 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-08T09:34:38.747 INFO:teuthology.lock.ops:Checking smithi151.front.sepia.ceph.com 2023-12-08T09:34:38.747 INFO:teuthology.lock.ops:New key found. Updating... 2023-12-08T09:34:38.765 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server 2023-12-08T09:34:38.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-12-08T09:34:38.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:34:38.832 DEBUG:teuthology.orchestra.run.smithi146:> true 2023-12-08T09:34:38.905 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2023-12-08T09:34:38.905 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2023-12-08T09:34:38.962 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2023-12-08T09:34:38.963 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2023-12-08T09:34:39.028 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2023-12-08T09:34:39.028 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="8" 2023-12-08T09:34:39.028 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2023-12-08T09:34:39.028 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2023-12-08T09:34:39.028 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="8" 2023-12-08T09:34:39.029 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el8" 2023-12-08T09:34:39.029 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-08T09:34:39.029 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2023-12-08T09:34:39.029 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-08T09:34:39.030 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2023-12-08T09:34:39.030 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T09:34:39.030 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-08T09:34:39.030 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T09:34:39.031 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2023-12-08T09:34:39.055 INFO:teuthology.lock.ops:Updating [smithi151.front.sepia.ceph.com]: set os type and version on server 2023-12-08T09:34:39.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2023-12-08T09:34:39.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T09:34:39.129 DEBUG:teuthology.orchestra.run.smithi151:> true 2023-12-08T09:34:39.200 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2023-12-08T09:34:39.200 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2023-12-08T09:34:39.267 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2023-12-08T09:34:39.267 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2023-12-08T09:34:39.327 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2023-12-08T09:34:39.327 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="8" 2023-12-08T09:34:39.327 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="8" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el8" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 8" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-08T09:34:39.328 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T09:34:39.329 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2023-12-08T09:34:39.364 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:34:39.464 INFO:teuthology.dispatcher.supervisor:Running job 7483388 2023-12-08T09:34:39.465 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-12-08_04:25:47-rados-wip-yuri5-testing-2023-12-07-0740-pacific-distro-default-smithi/7483388 --name yuriw-2023-12-08_04:25:47-rados-wip-yuri5-testing-2023-12-07-0740-pacific-distro-default-smithi --description rados/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} -- /home/teuthworker/archive/yuriw-2023-12-08_04:25:47-rados-wip-yuri5-testing-2023-12-07-0740-pacific-distro-default-smithi/7483388/orig.config.yaml 2023-12-08T09:34:39.467 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2023-12-08_04:25:47-rados-wip-yuri5-testing-2023-12-07-0740-pacific-distro-default-smithi/7483388 2023-12-08T09:34:39.468 INFO:teuthology.dispatcher.supervisor:Job PID: 87473 2023-12-08T09:34:39.468 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2023-12-08T09:36:39.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:38:39.764 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:40:39.927 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:42:40.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:44:40.279 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:46:40.455 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:48:40.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:50:40.864 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:52:41.036 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:54:41.216 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:56:41.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T09:58:41.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T10:00:41.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T10:02:41.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T10:02:41.969 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1 2023-12-08T10:02:42.085 INFO:teuthology.dispatcher.supervisor:Nuking machines... 2023-12-08T10:02:42.085 INFO:teuthology.nuke:Checking targets against current locks 2023-12-08T10:02:42.137 DEBUG:teuthology.nuke:shortname: smithi146 2023-12-08T10:02:42.138 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-08T10:02:42.158 DEBUG:teuthology.nuke:shortname: smithi151 2023-12-08T10:02:42.158 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-08T10:02:42.161 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-08_04:25:47-rados-wip-yuri5-testing-2023-12-07-0740-pacific-distro-default-smithi/7483388', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-08 09:29:19.201194', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDluQSNu72i5I1b8zxS0cSLumi7SCooD/y2CWC+jaTwr9KIsya8eSkbNqx/9VA+DgDZsi9ww+1C3mEp272lFuaY='} 2023-12-08T10:02:42.189 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-08_04:25:47-rados-wip-yuri5-testing-2023-12-07-0740-pacific-distro-default-smithi/7483388', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-08 09:29:19.203061', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFYIxSzPSDkPUnnrSZlm9G2xFP0kf1K/ssm+/5PhLCxMAC/Fw8AiZnDz4WOxKpHRa2fPW4erwkeiQIeEMHyAuC8='} 2023-12-08T10:02:42.214 INFO:teuthology.orchestra.console.smithi146:Power off 2023-12-08T10:02:42.214 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-08T10:02:42.236 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2023-12-08T10:02:42.236 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T10:02:42.251 INFO:teuthology.orchestra.console.smithi151:Power off 2023-12-08T10:02:42.252 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-08T10:02:42.260 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2023-12-08T10:02:42.273 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off 2023-12-08T10:02:42.273 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-08T10:02:42.302 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2023-12-08T10:02:42.361 INFO:teuthology.orchestra.console.smithi146:Power off completed 2023-12-08T10:02:42.403 INFO:teuthology.orchestra.console.smithi151:Power off completed 2023-12-08T10:02:42.510 INFO:teuthology.lock.ops:unlocked: smithi146.front.sepia.ceph.com 2023-12-08T10:02:42.552 INFO:teuthology.lock.ops:unlocked: smithi151.front.sepia.ceph.com