2024-04-17T08:44:34.998 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-17T08:44:34.999 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T08:44:35.175 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb6740a7c10>, signals=[15]) 2024-04-17T08:44:35.175 INFO:teuthology.lock.ops:Start node 'smithi033.front.sepia.ceph.com' reimaging 2024-04-17T08:44:35.175 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: reset os type and version on server 2024-04-17T08:44:35.175 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-17T08:44:35.202 INFO:teuthology.lock.ops:Start node 'smithi104.front.sepia.ceph.com' reimaging 2024-04-17T08:44:35.203 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: reset os type and version on server 2024-04-17T08:44:35.203 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-04-17T08:44:35.226 INFO:teuthology.lock.ops:Start node 'smithi167.front.sepia.ceph.com' reimaging 2024-04-17T08:44:35.227 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: reset os type and version on server 2024-04-17T08:44:35.227 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-04-17T08:44:35.316 DEBUG:teuthology.provision.fog.smithi033:Requesting image smithi_centos_9.stream (ID 10) 2024-04-17T08:44:35.356 DEBUG:teuthology.provision.fog.smithi167:Requesting image smithi_centos_9.stream (ID 10) 2024-04-17T08:44:35.364 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of centos 9.stream 2024-04-17T08:44:35.366 DEBUG:teuthology.provision.fog.smithi104:Requesting image smithi_centos_9.stream (ID 10) 2024-04-17T08:44:35.397 INFO:teuthology.provision.fog.smithi167:Scheduling deploy of centos 9.stream 2024-04-17T08:44:35.408 INFO:teuthology.provision.fog.smithi104:Scheduling deploy of centos 9.stream 2024-04-17T08:44:35.603 INFO:teuthology.orchestra.console.smithi033:Power off 2024-04-17T08:44:35.603 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T08:44:35.618 INFO:teuthology.orchestra.console.smithi167:Power off 2024-04-17T08:44:35.618 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T08:44:35.626 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-04-17T08:44:35.627 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:35.641 INFO:teuthology.orchestra.console.smithi104:Power off 2024-04-17T08:44:35.641 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-17T08:44:35.653 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-17T08:44:35.653 DEBUG:teuthology.orchestra.console.smithi167:power off output: Chassis Power Control: Down/Off 2024-04-17T08:44:35.653 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:35.665 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off 2024-04-17T08:44:35.666 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:35.675 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2024-04-17T08:44:35.686 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-04-17T08:44:35.754 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-04-17T08:44:35.776 INFO:teuthology.orchestra.console.smithi167:Power off completed 2024-04-17T08:44:35.788 INFO:teuthology.orchestra.console.smithi104:Power off completed 2024-04-17T08:44:35.855 INFO:teuthology.orchestra.console.smithi033:Power on 2024-04-17T08:44:35.856 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-17T08:44:35.877 INFO:teuthology.orchestra.console.smithi167:Power on 2024-04-17T08:44:35.878 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-17T08:44:35.887 DEBUG:teuthology.orchestra.console.smithi033:power on output: Chassis Power Control: Up/On 2024-04-17T08:44:35.887 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:35.895 INFO:teuthology.orchestra.console.smithi104:Power on 2024-04-17T08:44:35.895 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-17T08:44:35.904 DEBUG:teuthology.orchestra.console.smithi167:power on output: Chassis Power Control: Up/On 2024-04-17T08:44:35.904 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:35.913 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-17T08:44:35.918 DEBUG:teuthology.orchestra.console.smithi104:power on output: Chassis Power Control: Up/On 2024-04-17T08:44:35.919 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:35.926 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is off 2024-04-17T08:44:35.939 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-04-17T08:44:39.918 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:39.928 DEBUG:teuthology.orchestra.console.smithi167:pexpect command: ipmitool -H smithi167.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:39.940 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:40.028 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-17T08:44:40.037 DEBUG:teuthology.orchestra.console.smithi167:check power output: Chassis Power is on 2024-04-17T08:44:40.049 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-04-17T08:44:40.138 INFO:teuthology.orchestra.console.smithi167:Power on completed 2024-04-17T08:44:40.150 INFO:teuthology.orchestra.console.smithi104:Power on completed 2024-04-17T08:44:40.239 INFO:teuthology.provision.fog.smithi167:Waiting for deploy to finish 2024-04-17T08:44:40.251 INFO:teuthology.provision.fog.smithi104:Waiting for deploy to finish 2024-04-17T08:44:44.030 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-17T08:44:44.140 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-17T08:44:44.240 INFO:teuthology.orchestra.console.smithi033:Power on completed 2024-04-17T08:44:44.341 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2024-04-17T08:47:00.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:03.582 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-17T08:47:09.588 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:11.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:12.638 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-17T08:47:14.622 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-04-17T08:47:18.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:20.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:21.694 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-17T08:47:23.678 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-04-17T08:47:26.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:27.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:29.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:29.730 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T08:47:30.750 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-17T08:47:32.734 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-04-17T08:47:35.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:36.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:38.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:38.814 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T08:47:39.806 WARNING:teuthology.provision.fog.smithi033:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-04-17T08:47:41.790 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-04-17T08:47:44.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:45.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:47.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:47.874 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T08:47:50.846 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-04-17T08:47:53.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:56.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:47:56.958 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T08:47:59.902 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-04-17T08:48:02.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:48:05.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:48:06.014 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T08:48:08.958 WARNING:teuthology.provision.fog.smithi104:[Errno None] Unable to connect to port 22 on 172.21.15.104 2024-04-17T08:48:12.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:48:14.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:48:15.070 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T08:48:21.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:48:24.126 WARNING:teuthology.provision.fog.smithi167:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-04-17T08:48:30.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:48:45.826 WARNING:teuthology.provision.fog.smithi033:timed out 2024-04-17T08:48:51.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:48:52.090 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-17T08:48:52.091 DEBUG:teuthology.orchestra.run.smithi033:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-17T08:48:52.438 INFO:teuthology.provision.fog.smithi033:Node is ready 2024-04-17T08:48:52.438 DEBUG:teuthology.orchestra.run.smithi033:> hostname 2024-04-17T08:48:52.453 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033.front.sepia.ceph.com 2024-04-17T08:48:52.453 DEBUG:teuthology.orchestra.run.smithi033:> grep smithi033.front.sepia.ceph.com /etc/hosts 2024-04-17T08:48:52.508 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33 smithi033.front.sepia.ceph.com smithi033 2024-04-17T08:48:52.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2024-04-17T08:48:52.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hosts 2024-04-17T08:48:52.673 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/172.21.15.33/172.21.15.33/g' /etc/hosts 2024-04-17T08:48:52.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi033.front.sepia.ceph.com/smithi033/g' /etc/hostname 2024-04-17T08:48:52.815 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2024-04-17T08:48:52.887 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-17T08:48:52.942 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T08:48:52.943 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T08:48:52.943 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T08:48:52.943 INFO:teuthology.provision.fog.smithi033:Deploy complete! 2024-04-17T08:48:52.945 DEBUG:teuthology.parallel:result is None 2024-04-17T08:49:14.987 WARNING:teuthology.provision.fog.smithi104:timed out 2024-04-17T08:49:20.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:49:21.291 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-17T08:49:21.291 DEBUG:teuthology.orchestra.run.smithi104:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-17T08:49:21.614 INFO:teuthology.provision.fog.smithi104:Node is ready 2024-04-17T08:49:21.614 DEBUG:teuthology.orchestra.run.smithi104:> hostname 2024-04-17T08:49:21.629 INFO:teuthology.orchestra.run.smithi104.stdout:smithi104.front.sepia.ceph.com 2024-04-17T08:49:21.629 DEBUG:teuthology.orchestra.run.smithi104:> grep smithi104.front.sepia.ceph.com /etc/hosts 2024-04-17T08:49:21.685 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.15.104 smithi104.front.sepia.ceph.com smithi104 2024-04-17T08:49:21.685 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104 2024-04-17T08:49:21.809 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hosts 2024-04-17T08:49:21.837 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/172.21.15.104/172.21.15.104/g' /etc/hosts 2024-04-17T08:49:21.905 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i -e 's/smithi104.front.sepia.ceph.com/smithi104/g' /etc/hostname 2024-04-17T08:49:21.971 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname smithi104 2024-04-17T08:49:22.038 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-04-17T08:49:22.092 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T08:49:22.093 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-04-17T08:49:22.093 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T08:49:22.093 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T08:49:22.093 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T08:49:22.093 INFO:teuthology.provision.fog.smithi104:Deploy complete! 2024-04-17T08:49:22.094 DEBUG:teuthology.parallel:result is None 2024-04-17T08:49:30.135 WARNING:teuthology.provision.fog.smithi167:timed out 2024-04-17T08:49:36.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:49:36.496 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-17T08:49:36.496 DEBUG:teuthology.orchestra.run.smithi167:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-17T08:49:36.822 INFO:teuthology.provision.fog.smithi167:Node is ready 2024-04-17T08:49:36.822 DEBUG:teuthology.orchestra.run.smithi167:> hostname 2024-04-17T08:49:36.836 INFO:teuthology.orchestra.run.smithi167.stdout:smithi167.front.sepia.ceph.com 2024-04-17T08:49:36.836 DEBUG:teuthology.orchestra.run.smithi167:> grep smithi167.front.sepia.ceph.com /etc/hosts 2024-04-17T08:49:36.891 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.15.167 smithi167.front.sepia.ceph.com smithi167 2024-04-17T08:49:36.892 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167 2024-04-17T08:49:36.996 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hosts 2024-04-17T08:49:37.024 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/172.21.15.167/172.21.15.167/g' /etc/hosts 2024-04-17T08:49:37.091 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i -e 's/smithi167.front.sepia.ceph.com/smithi167/g' /etc/hostname 2024-04-17T08:49:37.158 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname smithi167 2024-04-17T08:49:37.224 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T08:49:37.279 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T08:49:37.280 INFO:teuthology.provision.fog.smithi167:Deploy complete! 2024-04-17T08:49:37.280 DEBUG:teuthology.parallel:result is None 2024-04-17T08:49:37.493 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com 2024-04-17T08:49:37.494 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-17T08:49:37.521 INFO:teuthology.lock.ops:Checking smithi104.front.sepia.ceph.com 2024-04-17T08:49:37.522 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-17T08:49:37.546 INFO:teuthology.lock.ops:Checking smithi167.front.sepia.ceph.com 2024-04-17T08:49:37.546 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-17T08:49:37.564 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server 2024-04-17T08:49:37.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-17T08:49:37.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:49:37.664 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-17T08:49:37.845 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-17T08:49:37.845 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-04-17T08:49:37.899 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-04-17T08:49:37.899 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T08:49:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-17T08:49:37.955 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T08:49:37.955 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T08:49:37.955 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T08:49:37.955 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-17T08:49:37.980 INFO:teuthology.lock.ops:Updating [smithi104.front.sepia.ceph.com]: set os type and version on server 2024-04-17T08:49:37.980 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-04-17T08:49:37.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:49:38.065 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-04-17T08:49:38.232 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-04-17T08:49:38.232 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-04-17T08:49:38.287 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-04-17T08:49:38.287 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T08:49:38.342 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-04-17T08:49:38.343 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T08:49:38.343 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T08:49:38.343 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T08:49:38.343 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-04-17T08:49:38.363 INFO:teuthology.lock.ops:Updating [smithi167.front.sepia.ceph.com]: set os type and version on server 2024-04-17T08:49:38.363 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T08:49:38.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-17T08:49:38.448 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-04-17T08:49:38.522 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-17T08:49:38.523 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-04-17T08:49:38.579 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-04-17T08:49:38.579 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-04-17T08:49:38.635 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-17T08:49:38.636 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-17T08:49:38.637 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-04-17T08:49:38.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T08:49:38.747 INFO:teuthology.dispatcher.supervisor:Running job 7659963 2024-04-17T08:49:38.747 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659963 --name rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/fsx}} -- /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659963/orig.config.yaml 2024-04-17T08:49:38.749 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/rishabh-2024-04-17_04:58:44-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7659963 2024-04-17T08:49:38.750 INFO:teuthology.dispatcher.supervisor:Job PID: 466700 2024-04-17T08:49:38.750 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-17T08:51:38.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T08:53:39.007 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T08:55:39.180 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T08:57:39.351 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-17T08:57:39.415 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1