2024-04-18T06:09:29.038 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-18T06:09:29.039 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:09:29.220 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8c0d266c10>, signals=[15]) 2024-04-18T06:09:29.220 INFO:teuthology.lock.ops:Start node 'smithi170.front.sepia.ceph.com' reimaging 2024-04-18T06:09:29.221 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T06:09:29.221 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-04-18T06:09:29.240 INFO:teuthology.lock.ops:Start node 'smithi194.front.sepia.ceph.com' reimaging 2024-04-18T06:09:29.240 INFO:teuthology.lock.ops:Updating [smithi194.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T06:09:29.240 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-04-18T06:09:29.264 INFO:teuthology.lock.ops:Start node 'smithi196.front.sepia.ceph.com' reimaging 2024-04-18T06:09:29.265 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T06:09:29.265 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-18T06:09:29.363 DEBUG:teuthology.provision.fog.smithi170:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-18T06:09:29.366 DEBUG:teuthology.provision.fog.smithi194:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-18T06:09:29.403 DEBUG:teuthology.provision.fog.smithi196:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-18T06:09:29.408 INFO:teuthology.provision.fog.smithi194:Scheduling deploy of rhel 8.6 2024-04-18T06:09:29.411 INFO:teuthology.provision.fog.smithi170:Scheduling deploy of rhel 8.6 2024-04-18T06:09:29.442 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of rhel 8.6 2024-04-18T06:09:29.907 INFO:teuthology.orchestra.console.smithi170:Power off 2024-04-18T06:09:29.907 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T06:09:29.935 INFO:teuthology.orchestra.console.smithi194:Power off 2024-04-18T06:09:29.935 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T06:09:29.943 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2024-04-18T06:09:29.943 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:29.956 DEBUG:teuthology.orchestra.console.smithi194:power off output: Chassis Power Control: Down/Off 2024-04-18T06:09:29.957 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:29.965 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-04-18T06:09:29.977 INFO:teuthology.orchestra.console.smithi196:Power off 2024-04-18T06:09:29.977 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T06:09:29.985 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is off 2024-04-18T06:09:29.998 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-04-18T06:09:29.998 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:30.021 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-18T06:09:30.066 INFO:teuthology.orchestra.console.smithi170:Power off completed 2024-04-18T06:09:30.087 INFO:teuthology.orchestra.console.smithi194:Power off completed 2024-04-18T06:09:30.121 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-04-18T06:09:30.167 INFO:teuthology.orchestra.console.smithi170:Power on 2024-04-18T06:09:30.167 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T06:09:30.188 INFO:teuthology.orchestra.console.smithi194:Power on 2024-04-18T06:09:30.188 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T06:09:30.196 DEBUG:teuthology.orchestra.console.smithi170:power on output: Chassis Power Control: Up/On 2024-04-18T06:09:30.197 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:30.211 DEBUG:teuthology.orchestra.console.smithi194:power on output: Chassis Power Control: Up/On 2024-04-18T06:09:30.212 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:30.220 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-04-18T06:09:30.222 INFO:teuthology.orchestra.console.smithi196:Power on 2024-04-18T06:09:30.223 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T06:09:30.244 DEBUG:teuthology.orchestra.console.smithi196:power on output: Chassis Power Control: Up/On 2024-04-18T06:09:30.244 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:30.260 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is off 2024-04-18T06:09:30.267 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-18T06:09:34.222 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:34.260 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:34.268 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:34.333 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-04-18T06:09:34.370 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is off 2024-04-18T06:09:34.377 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-18T06:09:34.434 INFO:teuthology.orchestra.console.smithi170:Power on completed 2024-04-18T06:09:34.478 INFO:teuthology.orchestra.console.smithi196:Power on completed 2024-04-18T06:09:34.535 INFO:teuthology.provision.fog.smithi170:Waiting for deploy to finish 2024-04-18T06:09:34.579 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2024-04-18T06:09:38.374 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T06:09:38.483 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-04-18T06:09:38.584 INFO:teuthology.orchestra.console.smithi194:Power on completed 2024-04-18T06:09:38.685 INFO:teuthology.provision.fog.smithi194:Waiting for deploy to finish 2024-04-18T06:14:53.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:14:57.022 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:15:03.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:06.082 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:15:12.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:15.166 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:15:21.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:24.222 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:15:28.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:30.225 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:31.198 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:15:33.278 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:15:37.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:39.280 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:40.258 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:15:42.338 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:15:46.263 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:48.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:49.346 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:15:55.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:15:58.430 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:16:04.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:06.686 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:16:07.486 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:16:12.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:13.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:15.742 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:16:16.546 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:16:21.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:22.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:24.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:24.802 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-18T06:16:27.678 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:16:30.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:31.506 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T06:16:31.506 DEBUG:teuthology.orchestra.run.smithi196:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T06:16:32.798 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:16:33.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:36.738 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:16:38.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:41.858 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:16:42.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:45.794 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:16:47.279 INFO:teuthology.provision.fog.smithi196:Node is ready 2024-04-18T06:16:47.280 DEBUG:teuthology.orchestra.run.smithi196:> hostname 2024-04-18T06:16:47.341 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com 2024-04-18T06:16:47.342 DEBUG:teuthology.orchestra.run.smithi196:> grep smithi196.front.sepia.ceph.com /etc/hosts 2024-04-18T06:16:47.400 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196 2024-04-18T06:16:47.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2024-04-18T06:16:47.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hosts 2024-04-18T06:16:47.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/172.21.15.196/172.21.15.196/g' /etc/hosts 2024-04-18T06:16:47.621 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hostname 2024-04-18T06:16:47.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2024-04-18T06:16:47.756 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-18T06:16:47.811 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:ID="rhel" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="fedora" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8.6" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T06:16:47.812 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T06:16:47.813 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T06:16:47.813 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T06:16:47.813 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T06:16:47.813 INFO:teuthology.provision.fog.smithi196:Deploy complete! 2024-04-18T06:16:47.815 DEBUG:teuthology.parallel:result is None 2024-04-18T06:16:47.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:50.910 WARNING:teuthology.provision.fog.smithi194:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-18T06:16:51.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:54.846 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:16:56.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:16:58.928 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T06:16:58.928 DEBUG:teuthology.orchestra.run.smithi194:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T06:17:00.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:17:03.902 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:17:09.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:17:12.958 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:17:18.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:17:19.191 INFO:teuthology.provision.fog.smithi194:Node is ready 2024-04-18T06:17:19.191 DEBUG:teuthology.orchestra.run.smithi194:> hostname 2024-04-18T06:17:19.210 INFO:teuthology.orchestra.run.smithi194.stdout:smithi194.front.sepia.ceph.com 2024-04-18T06:17:19.211 DEBUG:teuthology.orchestra.run.smithi194:> grep smithi194.front.sepia.ceph.com /etc/hosts 2024-04-18T06:17:19.268 INFO:teuthology.orchestra.run.smithi194.stdout:172.21.15.194 smithi194.front.sepia.ceph.com smithi194 2024-04-18T06:17:19.268 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname smithi194 2024-04-18T06:17:19.387 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i -e 's/smithi194.front.sepia.ceph.com/smithi194/g' /etc/hosts 2024-04-18T06:17:19.416 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i -e 's/172.21.15.194/172.21.15.194/g' /etc/hosts 2024-04-18T06:17:19.484 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i -e 's/smithi194.front.sepia.ceph.com/smithi194/g' /etc/hostname 2024-04-18T06:17:19.551 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname smithi194 2024-04-18T06:17:19.618 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:ID="rhel" 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="fedora" 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="8.6" 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el8" 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-04-18T06:17:19.675 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T06:17:19.676 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T06:17:19.676 INFO:teuthology.orchestra.run.smithi194.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T06:17:19.676 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T06:17:19.676 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-18T06:17:19.676 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T06:17:19.676 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T06:17:19.676 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T06:17:19.676 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T06:17:19.676 INFO:teuthology.provision.fog.smithi194:Deploy complete! 2024-04-18T06:17:19.677 DEBUG:teuthology.parallel:result is None 2024-04-18T06:17:37.310 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:17:43.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:17:46.366 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:17:52.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:17:55.422 WARNING:teuthology.provision.fog.smithi170:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-04-18T06:18:01.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:18:02.363 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T06:18:02.364 DEBUG:teuthology.orchestra.run.smithi170:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T06:18:18.050 INFO:teuthology.provision.fog.smithi170:Node is ready 2024-04-18T06:18:18.050 DEBUG:teuthology.orchestra.run.smithi170:> hostname 2024-04-18T06:18:18.073 INFO:teuthology.orchestra.run.smithi170.stdout:smithi170.front.sepia.ceph.com 2024-04-18T06:18:18.073 DEBUG:teuthology.orchestra.run.smithi170:> grep smithi170.front.sepia.ceph.com /etc/hosts 2024-04-18T06:18:18.133 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170 smithi170.front.sepia.ceph.com smithi170 2024-04-18T06:18:18.133 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname smithi170 2024-04-18T06:18:18.268 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hosts 2024-04-18T06:18:18.299 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/172.21.15.170/172.21.15.170/g' /etc/hosts 2024-04-18T06:18:18.368 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i -e 's/smithi170.front.sepia.ceph.com/smithi170/g' /etc/hostname 2024-04-18T06:18:18.437 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname smithi170 2024-04-18T06:18:18.505 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-04-18T06:18:18.562 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T06:18:18.562 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.6" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T06:18:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T06:18:18.564 INFO:teuthology.provision.fog.smithi170:Deploy complete! 2024-04-18T06:18:18.564 DEBUG:teuthology.parallel:result is None 2024-04-18T06:18:18.754 INFO:teuthology.lock.ops:Checking smithi170.front.sepia.ceph.com 2024-04-18T06:18:18.755 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T06:18:18.776 INFO:teuthology.lock.ops:Checking smithi194.front.sepia.ceph.com 2024-04-18T06:18:18.776 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T06:18:18.792 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com 2024-04-18T06:18:18.792 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T06:18:18.810 INFO:teuthology.lock.ops:Updating [smithi170.front.sepia.ceph.com]: set os type and version on server 2024-04-18T06:18:18.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-04-18T06:18:18.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:18:18.883 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-04-18T06:18:18.958 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-04-18T06:18:18.958 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-04-18T06:18:19.017 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-04-18T06:18:19.017 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.6" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T06:18:19.076 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T06:18:19.077 INFO:teuthology.orchestra.run.smithi170.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T06:18:19.077 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T06:18:19.077 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-18T06:18:19.077 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T06:18:19.077 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T06:18:19.077 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T06:18:19.077 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T06:18:19.078 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-04-18T06:18:19.101 INFO:teuthology.lock.ops:Updating [smithi194.front.sepia.ceph.com]: set os type and version on server 2024-04-18T06:18:19.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-18T06:18:19.101 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:18:19.176 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-04-18T06:18:19.316 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-18T06:18:19.316 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-04-18T06:18:19.372 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-04-18T06:18:19.373 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-04-18T06:18:19.429 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T06:18:19.429 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:ID="rhel" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="fedora" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="8.6" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el8" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T06:18:19.430 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T06:18:19.431 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T06:18:19.431 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T06:18:19.431 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-04-18T06:18:19.451 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server 2024-04-18T06:18:19.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-18T06:18:19.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T06:18:19.526 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-18T06:18:19.621 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-18T06:18:19.621 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-18T06:18:19.679 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-18T06:18:19.679 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-18T06:18:19.736 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T06:18:19.736 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T06:18:19.736 INFO:teuthology.orchestra.run.smithi196.stdout:ID="rhel" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="fedora" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8.6" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-18T06:18:19.737 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T06:18:19.738 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T06:18:19.738 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T06:18:19.738 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T06:18:19.738 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-18T06:18:19.759 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:18:19.863 INFO:teuthology.dispatcher.supervisor:Running job 7661657 2024-04-18T06:18:19.863 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661657 --name teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/kernel_untar_build}} -- /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661657/orig.config.yaml 2024-04-18T06:18:19.866 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661657 2024-04-18T06:18:19.866 INFO:teuthology.dispatcher.supervisor:Job PID: 1914082 2024-04-18T06:18:19.867 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-18T06:20:19.968 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:22:20.150 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:24:20.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:26:20.495 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:28:20.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:30:20.850 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:32:21.016 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:34:21.180 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:36:21.358 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:38:21.464 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T06:38:21.538 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1