2024-03-04T13:05:11.595 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T13:05:11.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:05:11.742 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8796dbcc10>, signals=[15]) 2024-03-04T13:05:11.743 INFO:teuthology.lock.ops:Start node 'smithi019.front.sepia.ceph.com' reimaging 2024-03-04T13:05:11.743 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: reset os type and version on server 2024-03-04T13:05:11.743 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-03-04T13:05:11.771 INFO:teuthology.lock.ops:Start node 'smithi037.front.sepia.ceph.com' reimaging 2024-03-04T13:05:11.771 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: reset os type and version on server 2024-03-04T13:05:11.771 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-03-04T13:05:11.886 DEBUG:teuthology.provision.fog.smithi019:Requesting image smithi_centos_9.stream (ID 10) 2024-03-04T13:05:11.913 DEBUG:teuthology.provision.fog.smithi037:Requesting image smithi_centos_9.stream (ID 10) 2024-03-04T13:05:11.937 INFO:teuthology.provision.fog.smithi019:Scheduling deploy of centos 9.stream 2024-03-04T13:05:11.962 INFO:teuthology.provision.fog.smithi037:Scheduling deploy of centos 9.stream 2024-03-04T13:05:12.162 INFO:teuthology.orchestra.console.smithi019:Power off 2024-03-04T13:05:12.163 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T13:05:12.176 INFO:teuthology.orchestra.console.smithi037:Power off 2024-03-04T13:05:12.176 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T13:05:12.184 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-03-04T13:05:12.185 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:12.200 DEBUG:teuthology.orchestra.console.smithi037:power off output: Chassis Power Control: Down/Off 2024-03-04T13:05:12.200 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:12.208 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-04T13:05:12.222 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-04T13:05:16.213 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:16.223 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:16.322 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-04T13:05:16.331 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-04T13:05:20.327 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:20.334 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:20.436 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-04T13:05:20.442 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-03-04T13:05:20.542 INFO:teuthology.orchestra.console.smithi037:Power off completed 2024-03-04T13:05:20.643 INFO:teuthology.orchestra.console.smithi037:Power on 2024-03-04T13:05:20.643 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-04T13:05:20.665 DEBUG:teuthology.orchestra.console.smithi037:power on output: Chassis Power Control: Up/On 2024-03-04T13:05:20.665 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:20.685 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-03-04T13:05:24.437 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:24.546 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-03-04T13:05:24.647 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-03-04T13:05:24.686 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:24.748 INFO:teuthology.orchestra.console.smithi019:Power on 2024-03-04T13:05:24.748 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-04T13:05:24.769 DEBUG:teuthology.orchestra.console.smithi019:power on output: Chassis Power Control: Up/On 2024-03-04T13:05:24.769 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:24.789 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-03-04T13:05:24.795 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is off 2024-03-04T13:05:28.791 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:28.798 DEBUG:teuthology.orchestra.console.smithi037:pexpect command: ipmitool -H smithi037.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T13:05:28.901 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-04T13:05:28.906 DEBUG:teuthology.orchestra.console.smithi037:check power output: Chassis Power is on 2024-03-04T13:05:29.002 INFO:teuthology.orchestra.console.smithi019:Power on completed 2024-03-04T13:05:29.007 INFO:teuthology.orchestra.console.smithi037:Power on completed 2024-03-04T13:05:29.103 INFO:teuthology.provision.fog.smithi019:Waiting for deploy to finish 2024-03-04T13:05:29.107 INFO:teuthology.provision.fog.smithi037:Waiting for deploy to finish 2024-03-04T13:07:44.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:07:47.868 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-03-04T13:07:53.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:07:56.956 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-03-04T13:07:59.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:02.940 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-04T13:08:02.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:06.016 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-03-04T13:08:08.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:11.996 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-04T13:08:12.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:15.068 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-03-04T13:08:18.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:21.055 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-04T13:08:21.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:24.124 WARNING:teuthology.provision.fog.smithi037:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-03-04T13:08:27.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:30.108 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-04T13:08:30.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:36.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:08:39.199 WARNING:teuthology.provision.fog.smithi019:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-04T13:08:45.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:09:30.172 WARNING:teuthology.provision.fog.smithi037:timed out 2024-03-04T13:09:36.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:09:36.446 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-04T13:09:36.446 DEBUG:teuthology.orchestra.run.smithi037:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-04T13:09:36.776 INFO:teuthology.provision.fog.smithi037:Node is ready 2024-03-04T13:09:36.776 DEBUG:teuthology.orchestra.run.smithi037:> hostname 2024-03-04T13:09:36.792 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037.front.sepia.ceph.com 2024-03-04T13:09:36.792 DEBUG:teuthology.orchestra.run.smithi037:> grep smithi037.front.sepia.ceph.com /etc/hosts 2024-03-04T13:09:36.848 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37 smithi037.front.sepia.ceph.com smithi037 2024-03-04T13:09:36.849 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname smithi037 2024-03-04T13:09:36.965 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/smithi037.front.sepia.ceph.com/smithi037/g' /etc/hosts 2024-03-04T13:09:36.997 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/172.21.15.37/172.21.15.37/g' /etc/hosts 2024-03-04T13:09:37.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i -e 's/smithi037.front.sepia.ceph.com/smithi037/g' /etc/hostname 2024-03-04T13:09:37.138 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname smithi037 2024-03-04T13:09:37.208 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-03-04T13:09:37.263 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-03-04T13:09:37.263 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-03-04T13:09:37.263 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-03-04T13:09:37.263 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-03-04T13:09:37.263 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T13:09:37.264 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T13:09:37.265 INFO:teuthology.provision.fog.smithi037:Deploy complete! 2024-03-04T13:09:37.266 DEBUG:teuthology.parallel:result is None 2024-03-04T13:09:45.207 WARNING:teuthology.provision.fog.smithi019:timed out 2024-03-04T13:09:51.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:09:51.459 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-04T13:09:51.459 DEBUG:teuthology.orchestra.run.smithi019:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-04T13:09:51.767 INFO:teuthology.provision.fog.smithi019:Node is ready 2024-03-04T13:09:51.767 DEBUG:teuthology.orchestra.run.smithi019:> hostname 2024-03-04T13:09:51.782 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019.front.sepia.ceph.com 2024-03-04T13:09:51.782 DEBUG:teuthology.orchestra.run.smithi019:> grep smithi019.front.sepia.ceph.com /etc/hosts 2024-03-04T13:09:51.837 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19 smithi019.front.sepia.ceph.com smithi019 2024-03-04T13:09:51.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname smithi019 2024-03-04T13:09:51.945 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/smithi019.front.sepia.ceph.com/smithi019/g' /etc/hosts 2024-03-04T13:09:51.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/172.21.15.19/172.21.15.19/g' /etc/hosts 2024-03-04T13:09:52.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i -e 's/smithi019.front.sepia.ceph.com/smithi019/g' /etc/hostname 2024-03-04T13:09:52.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname smithi019 2024-03-04T13:09:52.193 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-03-04T13:09:52.248 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T13:09:52.249 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T13:09:52.249 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T13:09:52.249 INFO:teuthology.provision.fog.smithi019:Deploy complete! 2024-03-04T13:09:52.249 DEBUG:teuthology.parallel:result is None 2024-03-04T13:09:52.421 INFO:teuthology.lock.ops:Checking smithi019.front.sepia.ceph.com 2024-03-04T13:09:52.422 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-04T13:09:52.439 INFO:teuthology.lock.ops:Checking smithi037.front.sepia.ceph.com 2024-03-04T13:09:52.440 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-04T13:09:52.457 INFO:teuthology.lock.ops:Updating [smithi019.front.sepia.ceph.com]: set os type and version on server 2024-03-04T13:09:52.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-04T13:09:52.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:09:52.559 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-03-04T13:09:52.627 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-04T13:09:52.627 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-03-04T13:09:52.682 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-03-04T13:09:52.683 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T13:09:52.738 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-03-04T13:09:52.739 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T13:09:52.739 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T13:09:52.739 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T13:09:52.740 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-03-04T13:09:52.765 INFO:teuthology.lock.ops:Updating [smithi037.front.sepia.ceph.com]: set os type and version on server 2024-03-04T13:09:52.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-04T13:09:52.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T13:09:52.870 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-03-04T13:09:53.055 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-03-04T13:09:53.055 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-03-04T13:09:53.110 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-03-04T13:09:53.110 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-03-04T13:09:53.165 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-03-04T13:09:53.165 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-03-04T13:09:53.165 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-03-04T13:09:53.165 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-03-04T13:09:53.165 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-04T13:09:53.166 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-04T13:09:53.166 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-03-04T13:09:53.190 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:09:53.274 INFO:teuthology.dispatcher.supervisor:Running job 7581156 2024-03-04T13:09:53.275 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-03-04_13:02:03-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7581156 --name vshankar-2024-03-04_13:02:03-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/vshankar-2024-03-04_13:02:03-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7581156/orig.config.yaml 2024-03-04T13:09:53.278 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-03-04_13:02:03-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7581156 2024-03-04T13:09:53.278 INFO:teuthology.dispatcher.supervisor:Job PID: 3782715 2024-03-04T13:09:53.278 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-04T13:11:53.376 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:13:53.552 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:15:53.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:17:53.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:19:54.033 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:21:54.141 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:23:54.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:25:54.468 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:27:54.653 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:29:54.824 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:31:54.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:33:55.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:35:55.348 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:37:55.512 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:39:55.665 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:41:55.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:43:56.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:45:56.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:47:56.335 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:49:56.603 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:51:56.756 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:53:56.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:55:56.989 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:57:57.152 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T13:59:57.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T14:01:57.464 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T14:03:57.625 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T14:05:57.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T14:07:57.948 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T14:07:58.013 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1