2024-03-16T07:52:46.725 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T07:52:46.726 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T07:52:46.899 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f58c7e2fc10>, signals=[15]) 2024-03-16T07:52:46.899 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging 2024-03-16T07:52:46.899 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server 2024-03-16T07:52:46.899 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-03-16T07:52:46.922 INFO:teuthology.lock.ops:Start node 'smithi134.front.sepia.ceph.com' reimaging 2024-03-16T07:52:46.922 INFO:teuthology.lock.ops:Updating [smithi134.front.sepia.ceph.com]: reset os type and version on server 2024-03-16T07:52:46.922 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-03-16T07:52:46.946 INFO:teuthology.lock.ops:Start node 'smithi186.front.sepia.ceph.com' reimaging 2024-03-16T07:52:46.946 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: reset os type and version on server 2024-03-16T07:52:46.946 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-03-16T07:52:47.046 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_centos_9.stream (ID 10) 2024-03-16T07:52:47.053 DEBUG:teuthology.provision.fog.smithi134:Requesting image smithi_centos_9.stream (ID 10) 2024-03-16T07:52:47.080 DEBUG:teuthology.provision.fog.smithi186:Requesting image smithi_centos_9.stream (ID 10) 2024-03-16T07:52:47.108 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of centos 9.stream 2024-03-16T07:52:47.111 INFO:teuthology.provision.fog.smithi134:Scheduling deploy of centos 9.stream 2024-03-16T07:52:47.122 INFO:teuthology.provision.fog.smithi186:Scheduling deploy of centos 9.stream 2024-03-16T07:52:47.314 INFO:teuthology.orchestra.console.smithi118:Power off 2024-03-16T07:52:47.314 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T07:52:47.328 INFO:teuthology.orchestra.console.smithi186:Power off 2024-03-16T07:52:47.328 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T07:52:47.338 INFO:teuthology.orchestra.console.smithi134:Power off 2024-03-16T07:52:47.338 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T07:52:47.346 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2024-03-16T07:52:47.347 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:47.354 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2024-03-16T07:52:47.354 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:47.362 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2024-03-16T07:52:47.362 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:47.370 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2024-03-16T07:52:47.377 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-03-16T07:52:47.386 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-03-16T07:52:51.377 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:51.384 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:51.390 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:51.487 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2024-03-16T07:52:51.493 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-03-16T07:52:51.498 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-03-16T07:52:55.490 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:55.497 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:55.504 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:55.599 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2024-03-16T07:52:55.606 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-03-16T07:52:55.612 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-03-16T07:52:55.712 INFO:teuthology.orchestra.console.smithi134:Power off completed 2024-03-16T07:52:55.813 INFO:teuthology.orchestra.console.smithi134:Power on 2024-03-16T07:52:55.814 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-16T07:52:55.836 DEBUG:teuthology.orchestra.console.smithi134:power on output: Chassis Power Control: Up/On 2024-03-16T07:52:55.836 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:55.856 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-03-16T07:52:59.602 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:59.609 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:59.712 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-03-16T07:52:59.718 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2024-03-16T07:52:59.813 INFO:teuthology.orchestra.console.smithi118:Power off completed 2024-03-16T07:52:59.819 INFO:teuthology.orchestra.console.smithi186:Power off completed 2024-03-16T07:52:59.857 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:59.914 INFO:teuthology.orchestra.console.smithi118:Power on 2024-03-16T07:52:59.915 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-16T07:52:59.922 INFO:teuthology.orchestra.console.smithi186:Power on 2024-03-16T07:52:59.922 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-16T07:52:59.936 DEBUG:teuthology.orchestra.console.smithi118:power on output: Chassis Power Control: Up/On 2024-03-16T07:52:59.936 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:59.944 DEBUG:teuthology.orchestra.console.smithi186:power on output: Chassis Power Control: Up/On 2024-03-16T07:52:59.945 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:52:59.956 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-03-16T07:52:59.965 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2024-03-16T07:52:59.966 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-03-16T07:53:03.958 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:53:03.966 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:53:03.972 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T07:53:04.067 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2024-03-16T07:53:04.074 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2024-03-16T07:53:04.081 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-03-16T07:53:04.168 INFO:teuthology.orchestra.console.smithi118:Power on completed 2024-03-16T07:53:04.175 INFO:teuthology.orchestra.console.smithi186:Power on completed 2024-03-16T07:53:04.182 INFO:teuthology.orchestra.console.smithi134:Power on completed 2024-03-16T07:53:04.269 INFO:teuthology.provision.fog.smithi118:Waiting for deploy to finish 2024-03-16T07:53:04.276 INFO:teuthology.provision.fog.smithi186:Waiting for deploy to finish 2024-03-16T07:53:04.283 INFO:teuthology.provision.fog.smithi134:Waiting for deploy to finish 2024-03-16T07:55:50.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:55:50.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:05.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:08.606 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2024-03-16T07:56:08.610 WARNING:teuthology.provision.fog.smithi134:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-03-16T07:56:14.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:14.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:17.662 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2024-03-16T07:56:17.662 WARNING:teuthology.provision.fog.smithi134:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-03-16T07:56:23.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:23.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:23.710 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-16T07:56:26.718 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2024-03-16T07:56:26.719 WARNING:teuthology.provision.fog.smithi134:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-03-16T07:56:29.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:32.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:32.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:32.766 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-16T07:56:35.774 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2024-03-16T07:56:35.775 WARNING:teuthology.provision.fog.smithi134:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-03-16T07:56:38.771 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:41.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:41.779 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:41.822 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-16T07:56:44.830 WARNING:teuthology.provision.fog.smithi186:[Errno None] Unable to connect to port 22 on 172.21.15.186 2024-03-16T07:56:47.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:50.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:50.878 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-16T07:56:56.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:56:59.966 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-16T07:57:05.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:57:09.054 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-03-16T07:57:15.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:57:41.806 WARNING:teuthology.provision.fog.smithi134:timed out 2024-03-16T07:57:47.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:57:48.037 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-16T07:57:48.038 DEBUG:teuthology.orchestra.run.smithi134:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-16T07:57:48.395 INFO:teuthology.provision.fog.smithi134:Node is ready 2024-03-16T07:57:48.395 DEBUG:teuthology.orchestra.run.smithi134:> hostname 2024-03-16T07:57:48.411 INFO:teuthology.orchestra.run.smithi134.stdout:smithi134.front.sepia.ceph.com 2024-03-16T07:57:48.411 DEBUG:teuthology.orchestra.run.smithi134:> grep smithi134.front.sepia.ceph.com /etc/hosts 2024-03-16T07:57:48.467 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.15.134 smithi134.front.sepia.ceph.com smithi134 2024-03-16T07:57:48.467 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname smithi134 2024-03-16T07:57:48.570 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i -e 's/smithi134.front.sepia.ceph.com/smithi134/g' /etc/hosts 2024-03-16T07:57:48.598 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i -e 's/172.21.15.134/172.21.15.134/g' /etc/hosts 2024-03-16T07:57:48.665 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i -e 's/smithi134.front.sepia.ceph.com/smithi134/g' /etc/hostname 2024-03-16T07:57:48.733 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname smithi134 2024-03-16T07:57:48.799 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-03-16T07:57:48.853 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-03-16T07:57:48.853 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-03-16T07:57:48.853 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-03-16T07:57:48.853 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:57:48.854 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:57:48.855 INFO:teuthology.provision.fog.smithi134:Deploy complete! 2024-03-16T07:57:48.856 DEBUG:teuthology.parallel:result is None 2024-03-16T07:57:50.838 WARNING:teuthology.provision.fog.smithi186:timed out 2024-03-16T07:57:56.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:57:57.067 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-16T07:57:57.067 DEBUG:teuthology.orchestra.run.smithi186:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-16T07:57:57.396 INFO:teuthology.provision.fog.smithi186:Node is ready 2024-03-16T07:57:57.396 DEBUG:teuthology.orchestra.run.smithi186:> hostname 2024-03-16T07:57:57.410 INFO:teuthology.orchestra.run.smithi186.stdout:smithi186.front.sepia.ceph.com 2024-03-16T07:57:57.410 DEBUG:teuthology.orchestra.run.smithi186:> grep smithi186.front.sepia.ceph.com /etc/hosts 2024-03-16T07:57:57.465 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.15.186 smithi186.front.sepia.ceph.com smithi186 2024-03-16T07:57:57.466 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname smithi186 2024-03-16T07:57:57.578 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/smithi186.front.sepia.ceph.com/smithi186/g' /etc/hosts 2024-03-16T07:57:57.606 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/172.21.15.186/172.21.15.186/g' /etc/hosts 2024-03-16T07:57:57.674 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/smithi186.front.sepia.ceph.com/smithi186/g' /etc/hostname 2024-03-16T07:57:57.740 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname smithi186 2024-03-16T07:57:57.807 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-03-16T07:57:57.862 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-03-16T07:57:57.862 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:57:57.863 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:57:57.864 INFO:teuthology.provision.fog.smithi186:Deploy complete! 2024-03-16T07:57:57.864 DEBUG:teuthology.parallel:result is None 2024-03-16T07:58:15.078 WARNING:teuthology.provision.fog.smithi118:timed out 2024-03-16T07:58:21.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:58:21.424 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-16T07:58:21.425 DEBUG:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-16T07:58:21.789 INFO:teuthology.provision.fog.smithi118:Node is ready 2024-03-16T07:58:21.789 DEBUG:teuthology.orchestra.run.smithi118:> hostname 2024-03-16T07:58:21.805 INFO:teuthology.orchestra.run.smithi118.stdout:smithi118.front.sepia.ceph.com 2024-03-16T07:58:21.806 DEBUG:teuthology.orchestra.run.smithi118:> grep smithi118.front.sepia.ceph.com /etc/hosts 2024-03-16T07:58:21.864 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.15.118 smithi118.front.sepia.ceph.com smithi118 2024-03-16T07:58:21.864 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2024-03-16T07:58:21.969 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hosts 2024-03-16T07:58:22.001 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/172.21.15.118/172.21.15.118/g' /etc/hosts 2024-03-16T07:58:22.073 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hostname 2024-03-16T07:58:22.145 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2024-03-16T07:58:22.215 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-03-16T07:58:22.271 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-03-16T07:58:22.271 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-03-16T07:58:22.271 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:58:22.272 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:58:22.273 INFO:teuthology.provision.fog.smithi118:Deploy complete! 2024-03-16T07:58:22.273 DEBUG:teuthology.parallel:result is None 2024-03-16T07:58:22.481 INFO:teuthology.lock.ops:Checking smithi118.front.sepia.ceph.com 2024-03-16T07:58:22.481 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-16T07:58:22.505 INFO:teuthology.lock.ops:Checking smithi134.front.sepia.ceph.com 2024-03-16T07:58:22.505 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-16T07:58:22.529 INFO:teuthology.lock.ops:Checking smithi186.front.sepia.ceph.com 2024-03-16T07:58:22.529 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-16T07:58:22.547 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: set os type and version on server 2024-03-16T07:58:22.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-16T07:58:22.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:58:22.634 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-03-16T07:58:22.701 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-03-16T07:58:22.701 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-03-16T07:58:22.756 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-03-16T07:58:22.756 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:58:22.811 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:58:22.812 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:58:22.812 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:58:22.812 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-03-16T07:58:22.830 INFO:teuthology.lock.ops:Updating [smithi134.front.sepia.ceph.com]: set os type and version on server 2024-03-16T07:58:22.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-16T07:58:22.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:58:22.918 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-03-16T07:58:23.084 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-03-16T07:58:23.084 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-03-16T07:58:23.139 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-03-16T07:58:23.139 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-03-16T07:58:23.194 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-03-16T07:58:23.194 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-03-16T07:58:23.194 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-03-16T07:58:23.194 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:58:23.195 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:58:23.196 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-03-16T07:58:23.222 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: set os type and version on server 2024-03-16T07:58:23.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-03-16T07:58:23.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T07:58:23.309 DEBUG:teuthology.orchestra.run.smithi186:> true 2024-03-16T07:58:23.477 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2024-03-16T07:58:23.477 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2024-03-16T07:58:23.532 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2024-03-16T07:58:23.532 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2024-03-16T07:58:23.586 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Stream" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="9" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="9" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el9" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:LOGO="fedora-logo-icon" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://centos.org/" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T07:58:23.587 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T07:58:23.588 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2024-03-16T07:58:23.613 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T07:58:23.718 INFO:teuthology.dispatcher.supervisor:Running job 7602278 2024-03-16T07:58:23.718 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602278 --name pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/fs/test_o_trunc}} -- /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602278/orig.config.yaml 2024-03-16T07:58:23.721 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602278 2024-03-16T07:58:23.722 INFO:teuthology.dispatcher.supervisor:Job PID: 1119680 2024-03-16T07:58:23.722 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-16T08:00:23.823 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:02:23.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:04:24.117 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:06:24.285 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:08:24.451 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:10:24.585 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:12:24.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:14:24.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:16:25.101 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:18:25.266 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:20:25.406 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:22:25.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:24:25.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:26:25.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:28:26.090 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:30:26.262 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:32:26.430 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:34:26.598 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:36:26.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:38:26.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:40:27.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:42:27.267 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T08:42:27.327 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1