2024-02-16T15:29:12.592 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-16T15:29:12.592 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:29:12.732 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9d69468c10>, signals=[15]) 2024-02-16T15:29:12.733 INFO:teuthology.lock.ops:Start node 'smithi073.front.sepia.ceph.com' reimaging 2024-02-16T15:29:12.733 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: reset os type and version on server 2024-02-16T15:29:12.733 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-02-16T15:29:12.756 INFO:teuthology.lock.ops:Start node 'smithi110.front.sepia.ceph.com' reimaging 2024-02-16T15:29:12.756 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: reset os type and version on server 2024-02-16T15:29:12.756 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-02-16T15:29:12.875 DEBUG:teuthology.provision.fog.smithi073:Requesting image smithi_centos_9.stream (ID 10) 2024-02-16T15:29:12.886 DEBUG:teuthology.provision.fog.smithi110:Requesting image smithi_centos_9.stream (ID 10) 2024-02-16T15:29:12.916 INFO:teuthology.provision.fog.smithi073:Scheduling deploy of centos 9.stream 2024-02-16T15:29:12.925 INFO:teuthology.provision.fog.smithi110:Scheduling deploy of centos 9.stream 2024-02-16T15:29:13.133 INFO:teuthology.orchestra.console.smithi073:Power off 2024-02-16T15:29:13.133 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T15:29:13.146 INFO:teuthology.orchestra.console.smithi110:Power off 2024-02-16T15:29:13.146 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-16T15:29:13.159 DEBUG:teuthology.orchestra.console.smithi073:power off output: Chassis Power Control: Down/Off 2024-02-16T15:29:13.160 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:13.167 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off 2024-02-16T15:29:13.167 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:13.182 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-02-16T15:29:13.188 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on 2024-02-16T15:29:17.187 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:17.195 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:17.297 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-02-16T15:29:17.303 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on 2024-02-16T15:29:21.302 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:21.308 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:21.411 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-02-16T15:29:21.416 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off 2024-02-16T15:29:21.517 INFO:teuthology.orchestra.console.smithi110:Power off completed 2024-02-16T15:29:21.617 INFO:teuthology.orchestra.console.smithi110:Power on 2024-02-16T15:29:21.618 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-16T15:29:21.638 DEBUG:teuthology.orchestra.console.smithi110:power on output: Chassis Power Control: Up/On 2024-02-16T15:29:21.639 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:21.660 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off 2024-02-16T15:29:25.412 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:25.521 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2024-02-16T15:29:25.621 INFO:teuthology.orchestra.console.smithi073:Power off completed 2024-02-16T15:29:25.661 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:25.723 INFO:teuthology.orchestra.console.smithi073:Power on 2024-02-16T15:29:25.723 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-16T15:29:25.747 DEBUG:teuthology.orchestra.console.smithi073:power on output: Chassis Power Control: Up/On 2024-02-16T15:29:25.748 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:25.768 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off 2024-02-16T15:29:25.768 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2024-02-16T15:29:29.773 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:29.780 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:29.883 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on 2024-02-16T15:29:29.889 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2024-02-16T15:29:29.984 INFO:teuthology.orchestra.console.smithi110:Power on completed 2024-02-16T15:29:30.085 INFO:teuthology.provision.fog.smithi110:Waiting for deploy to finish 2024-02-16T15:29:33.893 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-16T15:29:34.002 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-02-16T15:29:34.102 INFO:teuthology.orchestra.console.smithi073:Power on completed 2024-02-16T15:29:34.203 INFO:teuthology.provision.fog.smithi073:Waiting for deploy to finish 2024-02-16T15:32:00.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:03.900 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-16T15:32:04.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:08.060 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-02-16T15:32:09.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:12.956 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-16T15:32:14.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:17.116 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-02-16T15:32:18.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:22.016 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-16T15:32:23.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:26.172 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-02-16T15:32:28.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:31.100 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-16T15:32:32.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:35.228 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-02-16T15:32:37.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:40.156 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-16T15:32:41.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:44.284 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-02-16T15:32:46.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:49.212 WARNING:teuthology.provision.fog.smithi110:[Errno None] Unable to connect to port 22 on 172.21.15.110 2024-02-16T15:32:50.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:53.340 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-02-16T15:32:55.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:32:59.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:33:02.396 WARNING:teuthology.provision.fog.smithi073:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-02-16T15:33:08.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:33:55.265 WARNING:teuthology.provision.fog.smithi110:timed out 2024-02-16T15:34:01.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:34:01.517 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-16T15:34:01.517 DEBUG:teuthology.orchestra.run.smithi110:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-16T15:34:01.840 INFO:teuthology.provision.fog.smithi110:Node is ready 2024-02-16T15:34:01.840 DEBUG:teuthology.orchestra.run.smithi110:> hostname 2024-02-16T15:34:01.854 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110.front.sepia.ceph.com 2024-02-16T15:34:01.855 DEBUG:teuthology.orchestra.run.smithi110:> grep smithi110.front.sepia.ceph.com /etc/hosts 2024-02-16T15:34:01.909 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110 smithi110.front.sepia.ceph.com smithi110 2024-02-16T15:34:01.910 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110 2024-02-16T15:34:02.019 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hosts 2024-02-16T15:34:02.048 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/172.21.15.110/172.21.15.110/g' /etc/hosts 2024-02-16T15:34:02.115 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i -e 's/smithi110.front.sepia.ceph.com/smithi110/g' /etc/hostname 2024-02-16T15:34:02.183 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname smithi110 2024-02-16T15:34:02.249 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-02-16T15:34:02.304 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-02-16T15:34:02.305 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T15:34:02.305 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-02-16T15:34:02.305 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T15:34:02.305 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T15:34:02.305 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T15:34:02.306 INFO:teuthology.provision.fog.smithi110:Deploy complete! 2024-02-16T15:34:02.307 DEBUG:teuthology.parallel:result is None 2024-02-16T15:34:08.411 WARNING:teuthology.provision.fog.smithi073:timed out 2024-02-16T15:34:14.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:34:14.737 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-16T15:34:14.737 DEBUG:teuthology.orchestra.run.smithi073:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-16T15:34:15.076 INFO:teuthology.provision.fog.smithi073:Node is ready 2024-02-16T15:34:15.076 DEBUG:teuthology.orchestra.run.smithi073:> hostname 2024-02-16T15:34:15.090 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073.front.sepia.ceph.com 2024-02-16T15:34:15.091 DEBUG:teuthology.orchestra.run.smithi073:> grep smithi073.front.sepia.ceph.com /etc/hosts 2024-02-16T15:34:15.146 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73 smithi073.front.sepia.ceph.com smithi073 2024-02-16T15:34:15.146 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname smithi073 2024-02-16T15:34:15.266 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/smithi073.front.sepia.ceph.com/smithi073/g' /etc/hosts 2024-02-16T15:34:15.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/172.21.15.73/172.21.15.73/g' /etc/hosts 2024-02-16T15:34:15.359 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i -e 's/smithi073.front.sepia.ceph.com/smithi073/g' /etc/hostname 2024-02-16T15:34:15.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname smithi073 2024-02-16T15:34:15.493 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-02-16T15:34:15.547 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-02-16T15:34:15.547 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-02-16T15:34:15.547 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T15:34:15.548 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T15:34:15.549 INFO:teuthology.provision.fog.smithi073:Deploy complete! 2024-02-16T15:34:15.549 DEBUG:teuthology.parallel:result is None 2024-02-16T15:34:15.718 INFO:teuthology.lock.ops:Checking smithi073.front.sepia.ceph.com 2024-02-16T15:34:15.718 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-16T15:34:15.741 INFO:teuthology.lock.ops:Checking smithi110.front.sepia.ceph.com 2024-02-16T15:34:15.742 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-16T15:34:15.759 INFO:teuthology.lock.ops:Updating [smithi073.front.sepia.ceph.com]: set os type and version on server 2024-02-16T15:34:15.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-02-16T15:34:15.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:34:15.845 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-02-16T15:34:15.909 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-02-16T15:34:15.909 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-02-16T15:34:15.963 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-02-16T15:34:15.963 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-02-16T15:34:16.017 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-02-16T15:34:16.017 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-02-16T15:34:16.017 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T15:34:16.018 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T15:34:16.019 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-02-16T15:34:16.041 INFO:teuthology.lock.ops:Updating [smithi110.front.sepia.ceph.com]: set os type and version on server 2024-02-16T15:34:16.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-02-16T15:34:16.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-16T15:34:16.130 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-02-16T15:34:16.302 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-02-16T15:34:16.302 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-02-16T15:34:16.357 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-02-16T15:34:16.358 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-02-16T15:34:16.412 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-02-16T15:34:16.412 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-02-16T15:34:16.412 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-02-16T15:34:16.412 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-02-16T15:34:16.412 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-02-16T15:34:16.412 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-02-16T15:34:16.412 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-16T15:34:16.413 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-02-16T15:34:16.413 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-02-16T15:34:16.413 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-16T15:34:16.413 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-02-16T15:34:16.413 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-16T15:34:16.413 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-16T15:34:16.413 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-16T15:34:16.413 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-02-16T15:34:16.433 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:34:16.518 INFO:teuthology.dispatcher.supervisor:Running job 7562435 2024-02-16T15:34:16.518 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-02-16_15:10:58-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562435 --name pdonnell-2024-02-16_15:10:58-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/pdonnell-2024-02-16_15:10:58-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562435/orig.config.yaml 2024-02-16T15:34:16.521 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2024-02-16_15:10:58-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7562435 2024-02-16T15:34:16.521 INFO:teuthology.dispatcher.supervisor:Job PID: 666363 2024-02-16T15:34:16.522 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-16T15:36:16.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:38:16.795 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:40:16.965 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:42:17.133 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:44:17.264 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:46:17.426 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:48:17.593 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:50:17.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:52:17.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:54:18.037 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:56:18.195 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T15:58:18.393 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:00:18.562 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:02:18.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:04:18.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:06:19.073 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:08:19.237 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:10:19.517 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:12:19.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:14:19.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:16:20.013 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:18:20.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:20:20.349 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:22:20.505 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:24:20.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:26:20.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:28:21.045 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:30:21.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:32:21.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:34:21.445 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:36:21.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:38:21.714 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-16T16:38:21.791 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1