2024-03-07T09:51:51.185 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-07T09:51:51.186 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T09:51:51.383 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fac5989bc10>, signals=[15]) 2024-03-07T09:51:51.384 INFO:teuthology.lock.ops:Start node 'smithi059.front.sepia.ceph.com' reimaging 2024-03-07T09:51:51.384 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: reset os type and version on server 2024-03-07T09:51:51.384 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-03-07T09:51:51.415 INFO:teuthology.lock.ops:Start node 'smithi143.front.sepia.ceph.com' reimaging 2024-03-07T09:51:51.415 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: reset os type and version on server 2024-03-07T09:51:51.416 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-03-07T09:51:51.440 INFO:teuthology.lock.ops:Start node 'smithi160.front.sepia.ceph.com' reimaging 2024-03-07T09:51:51.440 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: reset os type and version on server 2024-03-07T09:51:51.440 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-03-07T09:51:51.524 DEBUG:teuthology.provision.fog.smithi059:Requesting image smithi_centos_9.stream (ID 10) 2024-03-07T09:51:51.547 DEBUG:teuthology.provision.fog.smithi143:Requesting image smithi_centos_9.stream (ID 10) 2024-03-07T09:51:51.579 INFO:teuthology.provision.fog.smithi059:Scheduling deploy of centos 9.stream 2024-03-07T09:51:51.588 DEBUG:teuthology.provision.fog.smithi160:Requesting image smithi_centos_9.stream (ID 10) 2024-03-07T09:51:51.599 INFO:teuthology.provision.fog.smithi143:Scheduling deploy of centos 9.stream 2024-03-07T09:51:51.631 INFO:teuthology.provision.fog.smithi160:Scheduling deploy of centos 9.stream 2024-03-07T09:51:51.800 INFO:teuthology.orchestra.console.smithi143:Power off 2024-03-07T09:51:51.800 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T09:51:51.819 INFO:teuthology.orchestra.console.smithi059:Power off 2024-03-07T09:51:51.819 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T09:51:51.831 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off 2024-03-07T09:51:51.831 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:51:51.841 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2024-03-07T09:51:51.842 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:51:51.858 INFO:teuthology.orchestra.console.smithi160:Power off 2024-03-07T09:51:51.858 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T09:51:51.866 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-03-07T09:51:51.867 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-03-07T09:51:51.879 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off 2024-03-07T09:51:51.879 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:51:51.900 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2024-03-07T09:51:55.869 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:51:55.876 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:51:55.901 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:51:55.979 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-03-07T09:51:55.985 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-03-07T09:51:56.011 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2024-03-07T09:51:59.980 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:51:59.986 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:00.012 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:00.088 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-03-07T09:52:00.095 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2024-03-07T09:52:00.121 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2024-03-07T09:52:00.190 INFO:teuthology.orchestra.console.smithi059:Power off completed 2024-03-07T09:52:00.196 INFO:teuthology.orchestra.console.smithi143:Power off completed 2024-03-07T09:52:00.290 INFO:teuthology.orchestra.console.smithi059:Power on 2024-03-07T09:52:00.291 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-07T09:52:00.298 INFO:teuthology.orchestra.console.smithi143:Power on 2024-03-07T09:52:00.298 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-07T09:52:00.312 DEBUG:teuthology.orchestra.console.smithi059:power on output: Chassis Power Control: Up/On 2024-03-07T09:52:00.312 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:00.321 DEBUG:teuthology.orchestra.console.smithi143:power on output: Chassis Power Control: Up/On 2024-03-07T09:52:00.321 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:00.333 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-03-07T09:52:00.341 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2024-03-07T09:52:04.126 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:04.235 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2024-03-07T09:52:04.334 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:04.341 INFO:teuthology.orchestra.console.smithi160:Power off completed 2024-03-07T09:52:04.342 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:04.443 INFO:teuthology.orchestra.console.smithi160:Power on 2024-03-07T09:52:04.443 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-07T09:52:04.451 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2024-03-07T09:52:04.451 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2024-03-07T09:52:04.464 DEBUG:teuthology.orchestra.console.smithi160:power on output: Chassis Power Control: Up/On 2024-03-07T09:52:04.465 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:04.486 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2024-03-07T09:52:08.453 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:08.460 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:08.487 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T09:52:08.569 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-03-07T09:52:08.594 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2024-03-07T09:52:08.649 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2024-03-07T09:52:08.669 INFO:teuthology.orchestra.console.smithi143:Power on completed 2024-03-07T09:52:08.695 INFO:teuthology.orchestra.console.smithi160:Power on completed 2024-03-07T09:52:08.750 INFO:teuthology.orchestra.console.smithi059:Power on completed 2024-03-07T09:52:08.771 INFO:teuthology.provision.fog.smithi143:Waiting for deploy to finish 2024-03-07T09:52:08.796 INFO:teuthology.provision.fog.smithi160:Waiting for deploy to finish 2024-03-07T09:52:08.851 INFO:teuthology.provision.fog.smithi059:Waiting for deploy to finish 2024-03-07T09:54:39.655 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:54:39.658 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:54:42.720 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-03-07T09:54:42.720 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-03-07T09:54:48.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:54:48.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:54:51.804 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-03-07T09:54:51.804 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-03-07T09:54:54.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:54:57.788 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T09:54:57.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:54:57.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:00.860 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-03-07T09:55:00.860 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-03-07T09:55:03.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:06.844 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T09:55:06.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:06.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:09.920 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-03-07T09:55:09.920 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-03-07T09:55:12.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:15.900 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T09:55:15.921 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:15.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:18.972 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-03-07T09:55:19.004 WARNING:teuthology.provision.fog.smithi059:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-03-07T09:55:21.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:24.956 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T09:55:24.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:25.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:28.028 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-03-07T09:55:30.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:34.016 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T09:55:34.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:37.084 WARNING:teuthology.provision.fog.smithi143:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-03-07T09:55:40.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:43.068 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T09:55:43.085 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:49.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:55:52.156 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T09:55:58.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:56:01.244 WARNING:teuthology.provision.fog.smithi160:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-03-07T09:56:07.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:56:25.024 WARNING:teuthology.provision.fog.smithi059:timed out 2024-03-07T09:56:31.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:56:31.285 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-07T09:56:31.286 DEBUG:teuthology.orchestra.run.smithi059:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-07T09:56:31.618 INFO:teuthology.provision.fog.smithi059:Node is ready 2024-03-07T09:56:31.618 DEBUG:teuthology.orchestra.run.smithi059:> hostname 2024-03-07T09:56:31.634 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059.front.sepia.ceph.com 2024-03-07T09:56:31.634 DEBUG:teuthology.orchestra.run.smithi059:> grep smithi059.front.sepia.ceph.com /etc/hosts 2024-03-07T09:56:31.690 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59 smithi059.front.sepia.ceph.com smithi059 2024-03-07T09:56:31.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2024-03-07T09:56:31.798 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hosts 2024-03-07T09:56:31.830 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/172.21.15.59/172.21.15.59/g' /etc/hosts 2024-03-07T09:56:31.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i -e 's/smithi059.front.sepia.ceph.com/smithi059/g' /etc/hostname 2024-03-07T09:56:31.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname smithi059 2024-03-07T09:56:32.043 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-03-07T09:56:32.098 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-03-07T09:56:32.098 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-03-07T09:56:32.098 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T09:56:32.099 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T09:56:32.100 INFO:teuthology.provision.fog.smithi059:Deploy complete! 2024-03-07T09:56:32.101 DEBUG:teuthology.parallel:result is None 2024-03-07T09:56:43.098 WARNING:teuthology.provision.fog.smithi143:timed out 2024-03-07T09:56:49.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:56:49.353 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-07T09:56:49.354 DEBUG:teuthology.orchestra.run.smithi143:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-07T09:56:49.696 INFO:teuthology.provision.fog.smithi143:Node is ready 2024-03-07T09:56:49.696 DEBUG:teuthology.orchestra.run.smithi143:> hostname 2024-03-07T09:56:49.711 INFO:teuthology.orchestra.run.smithi143.stdout:smithi143.front.sepia.ceph.com 2024-03-07T09:56:49.711 DEBUG:teuthology.orchestra.run.smithi143:> grep smithi143.front.sepia.ceph.com /etc/hosts 2024-03-07T09:56:49.766 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143 smithi143.front.sepia.ceph.com smithi143 2024-03-07T09:56:49.766 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname smithi143 2024-03-07T09:56:49.865 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i -e 's/smithi143.front.sepia.ceph.com/smithi143/g' /etc/hosts 2024-03-07T09:56:49.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i -e 's/172.21.15.143/172.21.15.143/g' /etc/hosts 2024-03-07T09:56:49.960 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i -e 's/smithi143.front.sepia.ceph.com/smithi143/g' /etc/hostname 2024-03-07T09:56:50.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname smithi143 2024-03-07T09:56:50.094 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-03-07T09:56:50.148 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-03-07T09:56:50.148 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-03-07T09:56:50.148 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-03-07T09:56:50.148 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-03-07T09:56:50.149 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-03-07T09:56:50.149 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-03-07T09:56:50.149 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T09:56:50.149 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-03-07T09:56:50.149 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-03-07T09:56:50.149 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T09:56:50.151 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-03-07T09:56:50.151 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T09:56:50.151 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T09:56:50.151 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T09:56:50.152 INFO:teuthology.provision.fog.smithi143:Deploy complete! 2024-03-07T09:56:50.152 DEBUG:teuthology.parallel:result is None 2024-03-07T09:57:07.266 WARNING:teuthology.provision.fog.smithi160:timed out 2024-03-07T09:57:13.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:57:13.619 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-07T09:57:13.619 DEBUG:teuthology.orchestra.run.smithi160:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-07T09:57:13.965 INFO:teuthology.provision.fog.smithi160:Node is ready 2024-03-07T09:57:13.965 DEBUG:teuthology.orchestra.run.smithi160:> hostname 2024-03-07T09:57:13.979 INFO:teuthology.orchestra.run.smithi160.stdout:smithi160.front.sepia.ceph.com 2024-03-07T09:57:13.980 DEBUG:teuthology.orchestra.run.smithi160:> grep smithi160.front.sepia.ceph.com /etc/hosts 2024-03-07T09:57:14.034 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.15.160 smithi160.front.sepia.ceph.com smithi160 2024-03-07T09:57:14.035 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2024-03-07T09:57:14.136 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hosts 2024-03-07T09:57:14.163 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/172.21.15.160/172.21.15.160/g' /etc/hosts 2024-03-07T09:57:14.230 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i -e 's/smithi160.front.sepia.ceph.com/smithi160/g' /etc/hostname 2024-03-07T09:57:14.297 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname smithi160 2024-03-07T09:57:14.363 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-03-07T09:57:14.417 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-03-07T09:57:14.417 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-03-07T09:57:14.417 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-03-07T09:57:14.417 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-03-07T09:57:14.417 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T09:57:14.418 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T09:57:14.418 INFO:teuthology.provision.fog.smithi160:Deploy complete! 2024-03-07T09:57:14.419 DEBUG:teuthology.parallel:result is None 2024-03-07T09:57:14.633 INFO:teuthology.lock.ops:Checking smithi059.front.sepia.ceph.com 2024-03-07T09:57:14.633 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-07T09:57:14.703 INFO:teuthology.lock.ops:Checking smithi143.front.sepia.ceph.com 2024-03-07T09:57:14.704 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-07T09:57:14.736 INFO:teuthology.lock.ops:Checking smithi160.front.sepia.ceph.com 2024-03-07T09:57:14.736 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-07T09:57:14.754 INFO:teuthology.lock.ops:Updating [smithi059.front.sepia.ceph.com]: set os type and version on server 2024-03-07T09:57:14.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-03-07T09:57:14.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:57:14.857 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-03-07T09:57:15.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-03-07T09:57:15.038 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-03-07T09:57:15.093 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-03-07T09:57:15.093 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-03-07T09:57:15.148 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-03-07T09:57:15.148 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-03-07T09:57:15.148 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-03-07T09:57:15.148 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-03-07T09:57:15.148 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-03-07T09:57:15.148 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-03-07T09:57:15.148 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T09:57:15.149 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-03-07T09:57:15.149 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-03-07T09:57:15.149 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T09:57:15.149 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-03-07T09:57:15.149 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T09:57:15.149 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T09:57:15.149 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T09:57:15.149 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-03-07T09:57:15.170 INFO:teuthology.lock.ops:Updating [smithi143.front.sepia.ceph.com]: set os type and version on server 2024-03-07T09:57:15.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-03-07T09:57:15.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:57:15.258 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-03-07T09:57:15.430 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-03-07T09:57:15.430 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-03-07T09:57:15.485 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-03-07T09:57:15.485 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-03-07T09:57:15.539 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-03-07T09:57:15.539 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T09:57:15.540 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T09:57:15.541 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-03-07T09:57:15.562 INFO:teuthology.lock.ops:Updating [smithi160.front.sepia.ceph.com]: set os type and version on server 2024-03-07T09:57:15.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T09:57:15.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T09:57:15.653 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-03-07T09:57:15.724 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-03-07T09:57:15.724 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-03-07T09:57:15.781 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-03-07T09:57:15.781 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-03-07T09:57:15.837 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-03-07T09:57:15.837 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-03-07T09:57:15.837 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-03-07T09:57:15.837 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-03-07T09:57:15.837 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-03-07T09:57:15.837 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-03-07T09:57:15.837 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T09:57:15.838 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-03-07T09:57:15.838 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-03-07T09:57:15.838 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T09:57:15.838 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-03-07T09:57:15.838 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T09:57:15.838 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T09:57:15.838 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T09:57:15.838 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-03-07T09:57:15.862 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T09:57:15.964 INFO:teuthology.dispatcher.supervisor:Running job 7584087 2024-03-07T09:57:15.965 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7584087 --name leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} -- /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7584087/orig.config.yaml 2024-03-07T09:57:15.967 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/leonidus-2024-03-07_05:46:12-fs-wip-lusov-qdb-distro-default-smithi/7584087 2024-03-07T09:57:15.967 INFO:teuthology.dispatcher.supervisor:Job PID: 3750339 2024-03-07T09:57:15.968 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-07T09:59:16.068 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:01:16.244 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:03:16.414 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:05:16.571 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:07:16.737 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:09:16.905 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:11:17.085 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:13:17.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:15:17.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:17:17.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:19:17.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:21:17.961 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:23:18.137 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:25:18.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:27:18.478 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:29:18.736 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:31:18.908 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:33:19.080 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:35:19.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:37:19.417 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:39:19.572 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T10:39:19.629 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1