2024-02-28T22:29:35.074 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-28T22:29:35.075 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:29:35.217 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efef809cc10>, signals=[15]) 2024-02-28T22:29:35.218 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging 2024-02-28T22:29:35.218 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server 2024-02-28T22:29:35.218 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2024-02-28T22:29:35.238 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging 2024-02-28T22:29:35.238 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server 2024-02-28T22:29:35.238 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-02-28T22:29:35.376 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_centos_9.stream (ID 10) 2024-02-28T22:29:35.380 DEBUG:teuthology.provision.fog.smithi104:Requesting image smithi_centos_9.stream (ID 10) 2024-02-28T22:29:35.424 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 9.stream 2024-02-28T22:29:35.427 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of centos 9.stream 2024-02-28T22:29:35.664 INFO:teuthology.orchestra.console.smithi104:Power off 2024-02-28T22:29:35.664 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-28T22:29:35.678 INFO:teuthology.orchestra.console.smithi023:Power off 2024-02-28T22:29:35.679 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-28T22:29:35.687 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off 2024-02-28T22:29:35.688 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:35.701 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off 2024-02-28T22:29:35.701 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:35.711 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-02-28T22:29:35.724 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-02-28T22:29:39.716 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:39.724 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:39.826 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-02-28T22:29:39.832 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-02-28T22:29:43.828 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:43.836 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:43.938 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-02-28T22:29:43.944 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-02-28T22:29:44.040 INFO:teuthology.orchestra.console.smithi104:Power off completed 2024-02-28T22:29:44.140 INFO:teuthology.orchestra.console.smithi104:Power on 2024-02-28T22:29:44.141 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-28T22:29:44.163 DEBUG:teuthology.orchestra.console.smithi104:power on output: Chassis Power Control: Up/On 2024-02-28T22:29:44.163 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:44.185 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-02-28T22:29:47.948 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:48.057 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2024-02-28T22:29:48.158 INFO:teuthology.orchestra.console.smithi023:Power off completed 2024-02-28T22:29:48.185 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:48.259 INFO:teuthology.orchestra.console.smithi023:Power on 2024-02-28T22:29:48.259 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-28T22:29:48.281 DEBUG:teuthology.orchestra.console.smithi023:power on output: Chassis Power Control: Up/On 2024-02-28T22:29:48.282 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:48.297 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-02-28T22:29:48.307 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2024-02-28T22:29:52.300 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:52.308 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-28T22:29:52.410 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-02-28T22:29:52.416 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-02-28T22:29:52.512 INFO:teuthology.orchestra.console.smithi104:Power on completed 2024-02-28T22:29:52.517 INFO:teuthology.orchestra.console.smithi023:Power on completed 2024-02-28T22:29:52.613 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish 2024-02-28T22:29:52.618 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish 2024-02-28T22:32:23.414 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:32:26.492 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-02-28T22:32:32.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:32:35.548 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-02-28T22:32:41.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:32:44.604 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-02-28T22:32:50.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:32:53.692 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-02-28T22:32:59.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:33:02.752 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-02-28T22:33:08.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:33:08.691 ERROR:teuthology.orchestra.connection:Error authenticating with smithi023.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-02-28T22:33:08.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:33:11.808 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-02-28T22:33:17.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:34:12.694 WARNING:teuthology.provision.fog.smithi023:timed out 2024-02-28T22:34:17.811 WARNING:teuthology.provision.fog.smithi104:timed out 2024-02-28T22:34:18.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:34:21.756 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-02-28T22:34:23.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:34:24.039 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-28T22:34:24.040 DEBUG:teuthology.orchestra.run.smithi104:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-28T22:34:24.386 INFO:teuthology.provision.fog.smithi104:Node is ready 2024-02-28T22:34:24.387 DEBUG:teuthology.orchestra.run.smithi104:> hostname 2024-02-28T22:34:24.401 INFO:teuthology.orchestra.run.smithi104.stdout:smithi104.front.sepia.ceph.com 2024-02-28T22:34:24.401 DEBUG:teuthology.orchestra.run.smithi104:> grep smithi104.front.sepia.ceph.com /etc/hosts 2024-02-28T22:34:24.456 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.104 smithi104.front.sepia.ceph.com smithi104 2024-02-28T22:34:24.456 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104 2024-02-28T22:34:24.582 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hosts 2024-02-28T22:34:24.609 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/172.21.15.104/172.21.15.104/g' /etc/hosts 2024-02-28T22:34:24.677 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hostname 2024-02-28T22:34:24.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104 2024-02-28T22:34:24.810 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-02-28T22:34:24.865 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-02-28T22:34:24.866 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-28T22:34:24.866 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-02-28T22:34:24.866 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-28T22:34:24.866 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-28T22:34:24.866 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-28T22:34:24.867 INFO:teuthology.provision.fog.smithi104:Deploy complete! 2024-02-28T22:34:24.868 DEBUG:teuthology.parallel:result is None 2024-02-28T22:34:27.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:34:30.812 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-02-28T22:34:36.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:34:39.868 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-02-28T22:34:45.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:35:45.871 WARNING:teuthology.provision.fog.smithi023:timed out 2024-02-28T22:35:51.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:35:52.106 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-28T22:35:52.107 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-28T22:35:52.425 INFO:teuthology.provision.fog.smithi023:Node is ready 2024-02-28T22:35:52.426 DEBUG:teuthology.orchestra.run.smithi023:> hostname 2024-02-28T22:35:52.440 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com 2024-02-28T22:35:52.441 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts 2024-02-28T22:35:52.496 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023 2024-02-28T22:35:52.497 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023 2024-02-28T22:35:52.609 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts 2024-02-28T22:35:52.640 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts 2024-02-28T22:35:52.710 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname 2024-02-28T22:35:52.780 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023 2024-02-28T22:35:52.850 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2024-02-28T22:35:52.905 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2024-02-28T22:35:52.906 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-28T22:35:52.907 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-28T22:35:52.907 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-28T22:35:52.907 INFO:teuthology.provision.fog.smithi023:Deploy complete! 2024-02-28T22:35:52.907 DEBUG:teuthology.parallel:result is None 2024-02-28T22:35:53.075 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com 2024-02-28T22:35:53.075 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-28T22:35:53.103 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com 2024-02-28T22:35:53.103 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-28T22:35:53.129 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server 2024-02-28T22:35:53.129 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-02-28T22:35:53.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:35:53.230 DEBUG:teuthology.orchestra.run.smithi023:> true 2024-02-28T22:35:53.298 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-02-28T22:35:53.298 DEBUG:teuthology.orchestra.run.smithi023:> uname -m 2024-02-28T22:35:53.353 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2024-02-28T22:35:53.353 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9" 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9" 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9" 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2024-02-28T22:35:53.409 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon" 2024-02-28T22:35:53.410 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-28T22:35:53.410 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2024-02-28T22:35:53.410 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-28T22:35:53.410 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-28T22:35:53.410 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-28T22:35:53.410 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2024-02-28T22:35:53.436 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server 2024-02-28T22:35:53.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-02-28T22:35:53.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-28T22:35:53.525 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-02-28T22:35:53.595 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-02-28T22:35:53.596 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-02-28T22:35:53.651 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-02-28T22:35:53.652 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-02-28T22:35:53.707 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-02-28T22:35:53.707 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-02-28T22:35:53.707 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-02-28T22:35:53.707 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-02-28T22:35:53.707 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-02-28T22:35:53.707 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-02-28T22:35:53.708 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-28T22:35:53.708 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-02-28T22:35:53.708 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-02-28T22:35:53.708 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-28T22:35:53.708 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-02-28T22:35:53.708 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-28T22:35:53.708 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-28T22:35:53.708 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-28T22:35:53.708 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-02-28T22:35:53.728 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:35:53.831 INFO:teuthology.dispatcher.supervisor:Running job 7575597 2024-02-28T22:35:53.831 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575597 --name vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575597/orig.config.yaml 2024-02-28T22:35:53.835 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-28_15:21:56-fs:upgrade:mds_upgrade_sequence-main-testing-default-smithi/7575597 2024-02-28T22:35:53.835 INFO:teuthology.dispatcher.supervisor:Job PID: 700110 2024-02-28T22:35:53.835 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-28T22:37:53.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:39:53.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:41:53.982 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:43:54.058 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:45:54.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:47:54.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:49:54.271 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:51:54.346 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:53:54.418 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:55:54.492 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:57:54.570 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T22:59:54.650 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T23:01:54.725 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T23:03:54.878 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T23:05:54.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T23:07:55.041 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-28T23:07:55.121 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1