2024-04-13T18:06:59.685 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-13T18:06:59.685 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:06:59.908 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb0efe14c10>, signals=[15]) 2024-04-13T18:06:59.908 INFO:teuthology.lock.ops:Start node 'smithi097.front.sepia.ceph.com' reimaging 2024-04-13T18:06:59.909 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: reset os type and version on server 2024-04-13T18:06:59.909 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-13T18:06:59.926 INFO:teuthology.lock.ops:Start node 'smithi105.front.sepia.ceph.com' reimaging 2024-04-13T18:06:59.926 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: reset os type and version on server 2024-04-13T18:06:59.926 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-04-13T18:06:59.951 INFO:teuthology.lock.ops:Start node 'smithi196.front.sepia.ceph.com' reimaging 2024-04-13T18:06:59.951 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: reset os type and version on server 2024-04-13T18:06:59.951 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-13T18:07:00.052 DEBUG:teuthology.provision.fog.smithi097:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-13T18:07:00.068 DEBUG:teuthology.provision.fog.smithi105:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-13T18:07:00.091 DEBUG:teuthology.provision.fog.smithi196:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-13T18:07:00.094 INFO:teuthology.provision.fog.smithi097:Scheduling deploy of rhel 8.6 2024-04-13T18:07:00.110 INFO:teuthology.provision.fog.smithi105:Scheduling deploy of rhel 8.6 2024-04-13T18:07:00.147 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of rhel 8.6 2024-04-13T18:07:00.421 INFO:teuthology.orchestra.console.smithi097:Power off 2024-04-13T18:07:00.422 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T18:07:00.447 INFO:teuthology.orchestra.console.smithi105:Power off 2024-04-13T18:07:00.448 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T18:07:00.457 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-04-13T18:07:00.458 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:00.480 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-04-13T18:07:00.481 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:00.495 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-04-13T18:07:00.527 INFO:teuthology.orchestra.console.smithi196:Power off 2024-04-13T18:07:00.527 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T18:07:00.539 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-04-13T18:07:00.558 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-04-13T18:07:00.558 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:00.597 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-04-13T18:07:00.599 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-13T18:07:00.640 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-04-13T18:07:00.698 INFO:teuthology.orchestra.console.smithi097:Power on 2024-04-13T18:07:00.698 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-13T18:07:00.707 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-04-13T18:07:00.726 DEBUG:teuthology.orchestra.console.smithi097:power on output: Chassis Power Control: Up/On 2024-04-13T18:07:00.726 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:00.743 INFO:teuthology.orchestra.console.smithi105:Power on 2024-04-13T18:07:00.743 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-13T18:07:00.759 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-04-13T18:07:00.774 DEBUG:teuthology.orchestra.console.smithi105:power on output: Chassis Power Control: Up/On 2024-04-13T18:07:00.774 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:00.801 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-04-13T18:07:00.808 INFO:teuthology.orchestra.console.smithi196:Power on 2024-04-13T18:07:00.808 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-13T18:07:00.836 DEBUG:teuthology.orchestra.console.smithi196:power on output: Chassis Power Control: Up/On 2024-04-13T18:07:00.836 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:00.871 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-13T18:07:04.762 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:04.802 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:04.873 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-13T18:07:04.874 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:04.913 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-04-13T18:07:04.975 INFO:teuthology.orchestra.console.smithi097:Power on completed 2024-04-13T18:07:04.985 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-13T18:07:05.076 INFO:teuthology.provision.fog.smithi097:Waiting for deploy to finish 2024-04-13T18:07:05.086 INFO:teuthology.orchestra.console.smithi196:Power on completed 2024-04-13T18:07:05.189 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2024-04-13T18:07:08.915 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:07:09.025 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-04-13T18:07:09.126 INFO:teuthology.orchestra.console.smithi105:Power on completed 2024-04-13T18:07:09.226 INFO:teuthology.provision.fog.smithi105:Waiting for deploy to finish 2024-04-13T18:09:40.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:09:43.678 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-13T18:09:49.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:09:52.734 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-13T18:09:58.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:10:01.790 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-13T18:10:07.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:10:10.846 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-13T18:10:16.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:10:19.934 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-13T18:10:25.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:10:28.990 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-13T18:10:34.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:10:36.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:10:37.193 ERROR:teuthology.orchestra.connection:Error authenticating with smithi196.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-04-13T18:10:38.050 WARNING:teuthology.provision.fog.smithi105:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-04-13T18:10:44.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:10:51.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:10:55.038 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:11:01.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:04.094 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:11:10.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:13.154 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:11:19.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:22.206 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:11:28.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:31.266 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:11:37.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:40.318 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:11:41.197 WARNING:teuthology.provision.fog.smithi196:timed out 2024-04-13T18:11:44.054 WARNING:teuthology.provision.fog.smithi105:timed out 2024-04-13T18:11:46.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:47.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:50.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:50.270 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-13T18:11:50.298 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-13T18:11:50.299 DEBUG:teuthology.orchestra.run.smithi105:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-13T18:11:50.529 INFO:teuthology.provision.fog.smithi105:Node is ready 2024-04-13T18:11:50.529 DEBUG:teuthology.orchestra.run.smithi105:> hostname 2024-04-13T18:11:50.591 INFO:teuthology.orchestra.run.smithi105.stdout:smithi105 2024-04-13T18:11:50.591 DEBUG:teuthology.orchestra.run.smithi105:> grep smithi105 /etc/hosts 2024-04-13T18:11:50.653 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.105 smithi105.front.sepia.ceph.com smithi105 2024-04-13T18:11:50.653 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname smithi105 2024-04-13T18:11:50.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/smithi105/smithi105/g' /etc/hosts 2024-04-13T18:11:50.802 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/172.21.15.105/172.21.15.105/g' /etc/hosts 2024-04-13T18:11:50.873 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i -e 's/smithi105/smithi105/g' /etc/hostname 2024-04-13T18:11:50.944 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname smithi105 2024-04-13T18:11:51.013 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:ID="rhel" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="fedora" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8.6" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:11:51.074 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:11:51.075 INFO:teuthology.orchestra.run.smithi105.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:11:51.075 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:11:51.075 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-13T18:11:51.075 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:11:51.075 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:11:51.075 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:11:51.075 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:11:51.076 INFO:teuthology.provision.fog.smithi105:Deploy complete! 2024-04-13T18:11:51.077 DEBUG:teuthology.parallel:result is None 2024-04-13T18:11:56.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:11:59.330 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-13T18:12:04.770 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:12:05.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:08.382 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-13T18:12:10.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:13.822 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:12:14.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:17.438 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-13T18:12:19.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:22.878 WARNING:teuthology.provision.fog.smithi097:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-04-13T18:12:23.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:23.679 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-13T18:12:23.679 DEBUG:teuthology.orchestra.run.smithi196:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-13T18:12:28.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:29.198 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-13T18:12:29.199 DEBUG:teuthology.orchestra.run.smithi097:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-13T18:12:43.925 INFO:teuthology.provision.fog.smithi196:Node is ready 2024-04-13T18:12:43.926 DEBUG:teuthology.orchestra.run.smithi196:> hostname 2024-04-13T18:12:43.948 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com 2024-04-13T18:12:43.949 DEBUG:teuthology.orchestra.run.smithi196:> grep smithi196.front.sepia.ceph.com /etc/hosts 2024-04-13T18:12:44.008 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196 2024-04-13T18:12:44.008 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2024-04-13T18:12:44.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hosts 2024-04-13T18:12:44.166 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/172.21.15.196/172.21.15.196/g' /etc/hosts 2024-04-13T18:12:44.234 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hostname 2024-04-13T18:12:44.302 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2024-04-13T18:12:44.369 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-13T18:12:44.426 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:12:44.426 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:12:44.426 INFO:teuthology.orchestra.run.smithi196.stdout:ID="rhel" 2024-04-13T18:12:44.426 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="fedora" 2024-04-13T18:12:44.426 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8.6" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:12:44.427 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:12:44.428 INFO:teuthology.provision.fog.smithi196:Deploy complete! 2024-04-13T18:12:44.428 DEBUG:teuthology.parallel:result is None 2024-04-13T18:12:45.258 INFO:teuthology.provision.fog.smithi097:Node is ready 2024-04-13T18:12:45.258 DEBUG:teuthology.orchestra.run.smithi097:> hostname 2024-04-13T18:12:45.277 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097.front.sepia.ceph.com 2024-04-13T18:12:45.277 DEBUG:teuthology.orchestra.run.smithi097:> grep smithi097.front.sepia.ceph.com /etc/hosts 2024-04-13T18:12:45.335 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97 smithi097.front.sepia.ceph.com smithi097 2024-04-13T18:12:45.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname smithi097 2024-04-13T18:12:45.445 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i -e 's/smithi097.front.sepia.ceph.com/smithi097/g' /etc/hosts 2024-04-13T18:12:45.473 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i -e 's/172.21.15.97/172.21.15.97/g' /etc/hosts 2024-04-13T18:12:45.540 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i -e 's/smithi097.front.sepia.ceph.com/smithi097/g' /etc/hostname 2024-04-13T18:12:45.607 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname smithi097 2024-04-13T18:12:45.675 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:12:45.732 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-13T18:12:45.733 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:12:45.733 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:12:45.733 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:12:45.733 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:12:45.733 INFO:teuthology.provision.fog.smithi097:Deploy complete! 2024-04-13T18:12:45.733 DEBUG:teuthology.parallel:result is None 2024-04-13T18:12:45.929 INFO:teuthology.lock.ops:Checking smithi097.front.sepia.ceph.com 2024-04-13T18:12:45.930 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-13T18:12:45.953 INFO:teuthology.lock.ops:Checking smithi105.front.sepia.ceph.com 2024-04-13T18:12:45.953 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-13T18:12:45.969 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com 2024-04-13T18:12:45.970 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-13T18:12:45.987 INFO:teuthology.lock.ops:Updating [smithi097.front.sepia.ceph.com]: set os type and version on server 2024-04-13T18:12:45.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-13T18:12:45.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:46.062 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-13T18:12:46.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-13T18:12:46.128 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-04-13T18:12:46.184 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-04-13T18:12:46.185 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-13T18:12:46.241 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:12:46.241 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:12:46.241 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2024-04-13T18:12:46.241 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2024-04-13T18:12:46.241 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2024-04-13T18:12:46.241 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:12:46.241 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:12:46.242 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:12:46.243 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-13T18:12:46.263 INFO:teuthology.lock.ops:Updating [smithi105.front.sepia.ceph.com]: set os type and version on server 2024-04-13T18:12:46.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-13T18:12:46.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:46.334 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-04-13T18:12:46.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-04-13T18:12:46.468 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-04-13T18:12:46.526 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-04-13T18:12:46.526 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-04-13T18:12:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:12:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:12:46.583 INFO:teuthology.orchestra.run.smithi105.stdout:ID="rhel" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="fedora" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8.6" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:12:46.584 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:12:46.585 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-04-13T18:12:46.645 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server 2024-04-13T18:12:46.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-13T18:12:46.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:12:46.731 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-13T18:12:46.807 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-13T18:12:46.807 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-13T18:12:46.866 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-13T18:12:46.867 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-13T18:12:46.925 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:12:46.925 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:12:46.925 INFO:teuthology.orchestra.run.smithi196.stdout:ID="rhel" 2024-04-13T18:12:46.925 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="fedora" 2024-04-13T18:12:46.925 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8.6" 2024-04-13T18:12:46.925 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:12:46.925 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:12:46.925 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:12:46.926 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:12:46.927 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-13T18:12:46.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:12:47.053 INFO:teuthology.dispatcher.supervisor:Running job 7651564 2024-04-13T18:12:47.053 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-10_22:24:02-fs-reef-distro-default-smithi/7651564 --name teuthology-2024-04-10_22:24:02-fs-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-workunit/postgres}} -- /home/teuthworker/archive/teuthology-2024-04-10_22:24:02-fs-reef-distro-default-smithi/7651564/orig.config.yaml 2024-04-13T18:12:47.056 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-10_22:24:02-fs-reef-distro-default-smithi/7651564 2024-04-13T18:12:47.057 INFO:teuthology.dispatcher.supervisor:Job PID: 1717939 2024-04-13T18:12:47.057 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-13T18:14:47.058 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:16:47.136 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:18:47.227 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:20:47.302 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:22:47.364 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:24:47.466 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:26:47.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:26:47.639 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1