2024-03-14T18:39:33.090 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-14T18:39:33.090 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T18:39:33.254 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbc4bbedc10>, signals=[15]) 2024-03-14T18:39:33.255 INFO:teuthology.lock.ops:Start node 'smithi032.front.sepia.ceph.com' reimaging 2024-03-14T18:39:33.255 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: reset os type and version on server 2024-03-14T18:39:33.255 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-03-14T18:39:33.271 INFO:teuthology.lock.ops:Start node 'smithi039.front.sepia.ceph.com' reimaging 2024-03-14T18:39:33.272 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: reset os type and version on server 2024-03-14T18:39:33.272 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-03-14T18:39:33.296 INFO:teuthology.lock.ops:Start node 'smithi100.front.sepia.ceph.com' reimaging 2024-03-14T18:39:33.296 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: reset os type and version on server 2024-03-14T18:39:33.296 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-03-14T18:39:33.392 DEBUG:teuthology.provision.fog.smithi032:Requesting image smithi_rhel_8.6 (ID 5) 2024-03-14T18:39:33.407 DEBUG:teuthology.provision.fog.smithi039:Requesting image smithi_rhel_8.6 (ID 5) 2024-03-14T18:39:33.433 INFO:teuthology.provision.fog.smithi032:Scheduling deploy of rhel 8.6 2024-03-14T18:39:33.436 DEBUG:teuthology.provision.fog.smithi100:Requesting image smithi_rhel_8.6 (ID 5) 2024-03-14T18:39:33.454 INFO:teuthology.provision.fog.smithi039:Scheduling deploy of rhel 8.6 2024-03-14T18:39:33.482 INFO:teuthology.provision.fog.smithi100:Scheduling deploy of rhel 8.6 2024-03-14T18:39:33.662 INFO:teuthology.orchestra.console.smithi032:Power off 2024-03-14T18:39:33.663 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T18:39:33.680 INFO:teuthology.orchestra.console.smithi039:Power off 2024-03-14T18:39:33.680 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T18:39:33.690 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2024-03-14T18:39:33.690 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:33.703 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2024-03-14T18:39:33.703 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:33.715 INFO:teuthology.orchestra.console.smithi100:Power off 2024-03-14T18:39:33.716 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-14T18:39:33.723 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-03-14T18:39:33.724 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-03-14T18:39:33.736 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-03-14T18:39:33.736 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:33.757 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-14T18:39:37.728 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:37.736 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:37.758 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:37.838 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-03-14T18:39:37.845 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-03-14T18:39:37.866 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-14T18:39:41.843 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:41.849 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:41.867 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:41.951 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2024-03-14T18:39:41.958 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-03-14T18:39:41.977 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-14T18:39:42.052 INFO:teuthology.orchestra.console.smithi032:Power off completed 2024-03-14T18:39:42.152 INFO:teuthology.orchestra.console.smithi032:Power on 2024-03-14T18:39:42.152 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-14T18:39:42.173 DEBUG:teuthology.orchestra.console.smithi032:power on output: Chassis Power Control: Up/On 2024-03-14T18:39:42.174 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:42.193 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2024-03-14T18:39:45.963 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:45.977 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:46.073 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-03-14T18:39:46.086 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-03-14T18:39:46.174 INFO:teuthology.orchestra.console.smithi039:Power off completed 2024-03-14T18:39:46.187 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-03-14T18:39:46.195 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:46.275 INFO:teuthology.orchestra.console.smithi039:Power on 2024-03-14T18:39:46.275 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-14T18:39:46.288 INFO:teuthology.orchestra.console.smithi100:Power on 2024-03-14T18:39:46.288 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-14T18:39:46.296 DEBUG:teuthology.orchestra.console.smithi039:power on output: Chassis Power Control: Up/On 2024-03-14T18:39:46.296 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:46.304 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2024-03-14T18:39:46.309 DEBUG:teuthology.orchestra.console.smithi100:power on output: Chassis Power Control: Up/On 2024-03-14T18:39:46.309 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:46.315 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-03-14T18:39:46.328 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-03-14T18:39:50.309 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:50.316 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:50.329 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:50.419 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-03-14T18:39:50.425 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2024-03-14T18:39:50.438 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-14T18:39:50.520 INFO:teuthology.orchestra.console.smithi032:Power on completed 2024-03-14T18:39:50.539 INFO:teuthology.orchestra.console.smithi100:Power on completed 2024-03-14T18:39:50.620 INFO:teuthology.provision.fog.smithi032:Waiting for deploy to finish 2024-03-14T18:39:50.640 INFO:teuthology.provision.fog.smithi100:Waiting for deploy to finish 2024-03-14T18:39:54.429 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-14T18:39:54.542 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2024-03-14T18:39:54.643 INFO:teuthology.orchestra.console.smithi039:Power on completed 2024-03-14T18:39:54.744 INFO:teuthology.provision.fog.smithi039:Waiting for deploy to finish 2024-03-14T18:42:06.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:42:06.637 ERROR:teuthology.orchestra.connection:Error authenticating with smithi032.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-14T18:42:36.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:42:54.878 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-14T18:42:55.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:42:55.649 ERROR:teuthology.orchestra.connection:Error authenticating with smithi039.front.sepia.ceph.com: AuthenticationException('Authentication failed.') 2024-03-14T18:43:00.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:03.934 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-14T18:43:09.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:09.982 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-14T18:43:10.640 WARNING:teuthology.provision.fog.smithi032:timed out 2024-03-14T18:43:12.990 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-14T18:43:15.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:16.641 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:18.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:19.038 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-14T18:43:22.046 WARNING:teuthology.provision.fog.smithi100:[Errno None] Unable to connect to port 22 on 172.21.15.100 2024-03-14T18:43:25.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:28.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:28.098 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-14T18:43:34.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:35.534 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-14T18:43:35.535 DEBUG:teuthology.orchestra.run.smithi032:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-14T18:43:37.154 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-14T18:43:43.156 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:46.206 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-14T18:43:51.232 INFO:teuthology.provision.fog.smithi032:Node is ready 2024-03-14T18:43:51.233 DEBUG:teuthology.orchestra.run.smithi032:> hostname 2024-03-14T18:43:51.255 INFO:teuthology.orchestra.run.smithi032.stdout:smithi045 2024-03-14T18:43:51.255 DEBUG:teuthology.orchestra.run.smithi032:> grep smithi045 /etc/hosts 2024-03-14T18:43:51.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-14T18:43:51.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i -e 's/smithi045/smithi032/g' /etc/hostname 2024-03-14T18:43:51.452 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname smithi032 2024-03-14T18:43:51.485 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.6 (Ootpa)" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.6" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2024-03-14T18:43:51.547 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-14T18:43:51.548 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T18:43:51.548 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-03-14T18:43:51.548 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-14T18:43:51.548 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-14T18:43:51.548 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-14T18:43:51.548 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-14T18:43:51.549 INFO:teuthology.provision.fog.smithi032:Deploy complete! 2024-03-14T18:43:51.553 DEBUG:teuthology.parallel:result is None 2024-03-14T18:43:52.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:43:55.263 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-14T18:44:01.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:44:04.318 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-14T18:44:10.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:44:13.378 WARNING:teuthology.provision.fog.smithi039:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-03-14T18:44:19.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:44:28.051 WARNING:teuthology.provision.fog.smithi100:timed out 2024-03-14T18:44:34.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:44:34.357 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-14T18:44:34.357 DEBUG:teuthology.orchestra.run.smithi100:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-14T18:44:34.599 INFO:teuthology.provision.fog.smithi100:Node is ready 2024-03-14T18:44:34.599 DEBUG:teuthology.orchestra.run.smithi100:> hostname 2024-03-14T18:44:34.657 INFO:teuthology.orchestra.run.smithi100.stdout:smithi100.front.sepia.ceph.com 2024-03-14T18:44:34.657 DEBUG:teuthology.orchestra.run.smithi100:> grep smithi100.front.sepia.ceph.com /etc/hosts 2024-03-14T18:44:34.715 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.15.100 smithi100.front.sepia.ceph.com smithi100 2024-03-14T18:44:34.715 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2024-03-14T18:44:34.829 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi100.front.sepia.ceph.com/smithi100/g' /etc/hosts 2024-03-14T18:44:34.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/172.21.15.100/172.21.15.100/g' /etc/hosts 2024-03-14T18:44:34.924 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i -e 's/smithi100.front.sepia.ceph.com/smithi100/g' /etc/hostname 2024-03-14T18:44:34.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname smithi100 2024-03-14T18:44:35.057 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Red Hat Enterprise Linux" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8.6 (Ootpa)" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:ID="rhel" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="fedora" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8.6" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.redhat.com/" 2024-03-14T18:44:35.114 INFO:teuthology.orchestra.run.smithi100.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-14T18:44:35.115 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T18:44:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-14T18:44:35.115 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-14T18:44:35.115 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-14T18:44:35.115 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-14T18:44:35.115 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-14T18:44:35.115 INFO:teuthology.provision.fog.smithi100:Deploy complete! 2024-03-14T18:44:35.116 DEBUG:teuthology.parallel:result is None 2024-03-14T18:45:19.383 WARNING:teuthology.provision.fog.smithi039:timed out 2024-03-14T18:45:25.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:45:25.708 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-14T18:45:25.708 DEBUG:teuthology.orchestra.run.smithi039:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-14T18:45:25.965 INFO:teuthology.provision.fog.smithi039:Node is ready 2024-03-14T18:45:25.965 DEBUG:teuthology.orchestra.run.smithi039:> hostname 2024-03-14T18:45:26.025 INFO:teuthology.orchestra.run.smithi039.stdout:smithi039.front.sepia.ceph.com 2024-03-14T18:45:26.025 DEBUG:teuthology.orchestra.run.smithi039:> grep smithi039.front.sepia.ceph.com /etc/hosts 2024-03-14T18:45:26.085 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.15.39 smithi039.front.sepia.ceph.com smithi039 2024-03-14T18:45:26.085 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname smithi039 2024-03-14T18:45:26.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/smithi039.front.sepia.ceph.com/smithi039/g' /etc/hosts 2024-03-14T18:45:26.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/172.21.15.39/172.21.15.39/g' /etc/hosts 2024-03-14T18:45:26.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i -e 's/smithi039.front.sepia.ceph.com/smithi039/g' /etc/hostname 2024-03-14T18:45:26.384 DEBUG:teuthology.orchestra.run.smithi039:> sudo hostname smithi039 2024-03-14T18:45:26.452 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-03-14T18:45:26.509 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Red Hat Enterprise Linux" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8.6 (Ootpa)" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:ID="rhel" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="fedora" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8.6" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.redhat.com/" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-14T18:45:26.510 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-14T18:45:26.511 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-14T18:45:26.511 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-14T18:45:26.511 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-14T18:45:26.511 INFO:teuthology.provision.fog.smithi039:Deploy complete! 2024-03-14T18:45:26.512 DEBUG:teuthology.parallel:result is None 2024-03-14T18:45:26.702 INFO:teuthology.lock.ops:Checking smithi032.front.sepia.ceph.com 2024-03-14T18:45:26.703 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-14T18:45:26.725 INFO:teuthology.lock.ops:Checking smithi039.front.sepia.ceph.com 2024-03-14T18:45:26.725 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-14T18:45:26.742 INFO:teuthology.lock.ops:Checking smithi100.front.sepia.ceph.com 2024-03-14T18:45:26.742 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-14T18:45:26.759 INFO:teuthology.lock.ops:Updating [smithi032.front.sepia.ceph.com]: set os type and version on server 2024-03-14T18:45:26.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-03-14T18:45:26.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:45:26.832 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-03-14T18:45:26.978 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-03-14T18:45:26.978 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-03-14T18:45:27.039 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-03-14T18:45:27.040 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Red Hat Enterprise Linux" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="8.6 (Ootpa)" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:ID="rhel" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="fedora" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="8.6" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el8" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.redhat.com/" 2024-03-14T18:45:27.101 INFO:teuthology.orchestra.run.smithi032.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-14T18:45:27.102 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T18:45:27.102 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-03-14T18:45:27.102 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-14T18:45:27.102 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-14T18:45:27.102 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-14T18:45:27.102 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-14T18:45:27.102 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-03-14T18:45:27.125 INFO:teuthology.lock.ops:Updating [smithi039.front.sepia.ceph.com]: set os type and version on server 2024-03-14T18:45:27.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-14T18:45:27.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:45:27.195 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-03-14T18:45:27.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-03-14T18:45:27.266 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-03-14T18:45:27.323 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-03-14T18:45:27.323 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Red Hat Enterprise Linux" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="8.6 (Ootpa)" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:ID="rhel" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="fedora" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="8.6" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el8" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-14T18:45:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.redhat.com/" 2024-03-14T18:45:27.381 INFO:teuthology.orchestra.run.smithi039.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-14T18:45:27.381 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T18:45:27.381 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-03-14T18:45:27.381 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-14T18:45:27.381 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-14T18:45:27.381 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-14T18:45:27.381 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-14T18:45:27.381 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-03-14T18:45:27.408 INFO:teuthology.lock.ops:Updating [smithi100.front.sepia.ceph.com]: set os type and version on server 2024-03-14T18:45:27.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-14T18:45:27.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-14T18:45:27.483 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-14T18:45:27.619 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-14T18:45:27.620 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-03-14T18:45:27.677 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-03-14T18:45:27.677 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-03-14T18:45:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="Red Hat Enterprise Linux" 2024-03-14T18:45:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="8.6 (Ootpa)" 2024-03-14T18:45:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:ID="rhel" 2024-03-14T18:45:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="fedora" 2024-03-14T18:45:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="8.6" 2024-03-14T18:45:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el8" 2024-03-14T18:45:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-14T18:45:27.734 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://www.redhat.com/" 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-14T18:45:27.735 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-14T18:45:27.736 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-03-14T18:45:27.758 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T18:45:27.864 INFO:teuthology.dispatcher.supervisor:Running job 7600316 2024-03-14T18:45:27.864 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600316 --name yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsx}} -- /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600316/orig.config.yaml 2024-03-14T18:45:27.870 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-03-14_15:28:28-fs-wip-yuri4-testing-2024-03-13-0733-reef-distro-default-smithi/7600316 2024-03-14T18:45:27.871 INFO:teuthology.dispatcher.supervisor:Job PID: 1900079 2024-03-14T18:45:27.871 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-14T18:47:27.873 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T18:49:27.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T18:51:28.029 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T18:53:28.101 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T18:55:28.166 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T18:57:28.230 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T18:59:28.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T19:01:28.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T19:03:28.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T19:05:28.511 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T19:07:28.608 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T19:09:28.681 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T19:11:28.753 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-14T19:11:28.827 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1