2024-03-13T18:39:25.747 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-13T18:39:25.747 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T18:39:25.930 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f39fe6e7c10>, signals=[15]) 2024-03-13T18:39:25.930 INFO:teuthology.lock.ops:Start node 'smithi080.front.sepia.ceph.com' reimaging 2024-03-13T18:39:25.931 INFO:teuthology.lock.ops:Updating [smithi080.front.sepia.ceph.com]: reset os type and version on server 2024-03-13T18:39:25.931 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-03-13T18:39:25.958 INFO:teuthology.lock.ops:Start node 'smithi087.front.sepia.ceph.com' reimaging 2024-03-13T18:39:25.958 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: reset os type and version on server 2024-03-13T18:39:25.958 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-03-13T18:39:25.974 INFO:teuthology.lock.ops:Start node 'smithi106.front.sepia.ceph.com' reimaging 2024-03-13T18:39:25.974 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: reset os type and version on server 2024-03-13T18:39:25.974 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-03-13T18:39:26.076 DEBUG:teuthology.provision.fog.smithi080:Requesting image smithi_rhel_8.6 (ID 5) 2024-03-13T18:39:26.105 DEBUG:teuthology.provision.fog.smithi106:Requesting image smithi_rhel_8.6 (ID 5) 2024-03-13T18:39:26.109 DEBUG:teuthology.provision.fog.smithi087:Requesting image smithi_rhel_8.6 (ID 5) 2024-03-13T18:39:26.114 INFO:teuthology.provision.fog.smithi080:Scheduling deploy of rhel 8.6 2024-03-13T18:39:26.158 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of rhel 8.6 2024-03-13T18:39:26.170 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of rhel 8.6 2024-03-13T18:39:26.529 INFO:teuthology.orchestra.console.smithi080:Power off 2024-03-13T18:39:26.529 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T18:39:26.544 INFO:teuthology.orchestra.console.smithi087:Power off 2024-03-13T18:39:26.544 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T18:39:26.555 INFO:teuthology.orchestra.console.smithi106:Power off 2024-03-13T18:39:26.556 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T18:39:26.564 DEBUG:teuthology.orchestra.console.smithi080:power off output: Chassis Power Control: Down/Off 2024-03-13T18:39:26.564 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:26.572 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2024-03-13T18:39:26.573 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:26.581 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-03-13T18:39:26.581 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:26.589 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is off 2024-03-13T18:39:26.592 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2024-03-13T18:39:26.602 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-03-13T18:39:26.690 INFO:teuthology.orchestra.console.smithi080:Power off completed 2024-03-13T18:39:26.693 INFO:teuthology.orchestra.console.smithi087:Power off completed 2024-03-13T18:39:26.703 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-03-13T18:39:26.792 INFO:teuthology.orchestra.console.smithi080:Power on 2024-03-13T18:39:26.792 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-13T18:39:26.800 INFO:teuthology.orchestra.console.smithi087:Power on 2024-03-13T18:39:26.800 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-13T18:39:26.807 INFO:teuthology.orchestra.console.smithi106:Power on 2024-03-13T18:39:26.807 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-03-13T18:39:26.815 DEBUG:teuthology.orchestra.console.smithi080:power on output: Chassis Power Control: Up/On 2024-03-13T18:39:26.815 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:26.823 DEBUG:teuthology.orchestra.console.smithi087:power on output: Chassis Power Control: Up/On 2024-03-13T18:39:26.824 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:26.831 DEBUG:teuthology.orchestra.console.smithi106:power on output: Chassis Power Control: Up/On 2024-03-13T18:39:26.832 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:26.839 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is off 2024-03-13T18:39:26.844 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2024-03-13T18:39:26.853 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-03-13T18:39:30.843 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:30.851 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:30.858 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:30.953 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is on 2024-03-13T18:39:30.959 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2024-03-13T18:39:30.966 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-03-13T18:39:31.054 INFO:teuthology.orchestra.console.smithi080:Power on completed 2024-03-13T18:39:31.061 INFO:teuthology.orchestra.console.smithi087:Power on completed 2024-03-13T18:39:31.155 INFO:teuthology.provision.fog.smithi080:Waiting for deploy to finish 2024-03-13T18:39:31.162 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish 2024-03-13T18:39:34.970 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T18:39:35.096 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-03-13T18:39:35.197 INFO:teuthology.orchestra.console.smithi106:Power on completed 2024-03-13T18:39:35.298 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish 2024-03-13T18:44:07.834 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:10.914 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:44:16.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:19.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:19.998 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:44:22.142 WARNING:teuthology.provision.fog.smithi080:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-13T18:44:26.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:28.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:29.054 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:44:31.198 WARNING:teuthology.provision.fog.smithi080:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-13T18:44:34.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:35.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:37.182 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-03-13T18:44:37.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:38.110 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:44:40.254 WARNING:teuthology.provision.fog.smithi080:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-13T18:44:43.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:44.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:46.238 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-03-13T18:44:46.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:47.167 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:44:49.310 WARNING:teuthology.provision.fog.smithi080:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-13T18:44:52.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:53.169 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:55.294 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-03-13T18:44:55.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:44:56.222 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:44:58.366 WARNING:teuthology.provision.fog.smithi080:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-13T18:45:01.299 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:02.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:04.350 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-03-13T18:45:04.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:07.422 WARNING:teuthology.provision.fog.smithi080:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-13T18:45:10.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:13.406 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-03-13T18:45:13.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:16.478 WARNING:teuthology.provision.fog.smithi080:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-13T18:45:19.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:20.542 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:45:22.462 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-03-13T18:45:22.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:26.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:28.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:29.598 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:45:31.518 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-03-13T18:45:35.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:37.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:38.654 WARNING:teuthology.provision.fog.smithi106:[Errno None] Unable to connect to port 22 on 172.21.15.106 2024-03-13T18:45:44.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:45:45.369 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-13T18:45:45.370 DEBUG:teuthology.orchestra.run.smithi106:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-13T18:46:00.936 INFO:teuthology.provision.fog.smithi106:Node is ready 2024-03-13T18:46:00.936 DEBUG:teuthology.orchestra.run.smithi106:> hostname 2024-03-13T18:46:00.958 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com 2024-03-13T18:46:00.958 DEBUG:teuthology.orchestra.run.smithi106:> grep smithi106.front.sepia.ceph.com /etc/hosts 2024-03-13T18:46:01.018 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106 2024-03-13T18:46:01.019 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname smithi106 2024-03-13T18:46:01.155 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/smithi106.front.sepia.ceph.com/smithi106/g' /etc/hosts 2024-03-13T18:46:01.189 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/172.21.15.106/172.21.15.106/g' /etc/hosts 2024-03-13T18:46:01.263 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/smithi106.front.sepia.ceph.com/smithi106/g' /etc/hostname 2024-03-13T18:46:01.336 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname smithi106 2024-03-13T18:46:01.410 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-03-13T18:46:01.470 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Red Hat Enterprise Linux" 2024-03-13T18:46:01.470 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8.6 (Ootpa)" 2024-03-13T18:46:01.470 INFO:teuthology.orchestra.run.smithi106.stdout:ID="rhel" 2024-03-13T18:46:01.470 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="fedora" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8.6" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.redhat.com/" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-13T18:46:01.471 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-13T18:46:01.472 INFO:teuthology.provision.fog.smithi106:Deploy complete! 2024-03-13T18:46:01.474 DEBUG:teuthology.parallel:result is None 2024-03-13T18:46:22.487 WARNING:teuthology.provision.fog.smithi080:timed out 2024-03-13T18:46:28.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:46:28.734 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-13T18:46:28.734 DEBUG:teuthology.orchestra.run.smithi080:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-13T18:46:29.029 INFO:teuthology.provision.fog.smithi080:Node is ready 2024-03-13T18:46:29.029 DEBUG:teuthology.orchestra.run.smithi080:> hostname 2024-03-13T18:46:29.048 INFO:teuthology.orchestra.run.smithi080.stdout:smithi080 2024-03-13T18:46:29.049 DEBUG:teuthology.orchestra.run.smithi080:> grep smithi080 /etc/hosts 2024-03-13T18:46:29.109 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.15.80 smithi080.front.sepia.ceph.com smithi080 2024-03-13T18:46:29.109 DEBUG:teuthology.orchestra.run.smithi080:> sudo hostname smithi080 2024-03-13T18:46:29.224 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i -e 's/smithi080/smithi080/g' /etc/hosts 2024-03-13T18:46:29.257 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i -e 's/172.21.15.80/172.21.15.80/g' /etc/hosts 2024-03-13T18:46:29.330 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i -e 's/smithi080/smithi080/g' /etc/hostname 2024-03-13T18:46:29.403 DEBUG:teuthology.orchestra.run.smithi080:> sudo hostname smithi080 2024-03-13T18:46:29.475 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-03-13T18:46:29.535 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="Red Hat Enterprise Linux" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="8.6 (Ootpa)" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:ID="rhel" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE="fedora" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="8.6" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:PLATFORM_ID="platform:el8" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:ANSI_COLOR="0;31" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.redhat.com/" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T18:46:29.536 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-13T18:46:29.537 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-13T18:46:29.537 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-13T18:46:29.537 INFO:teuthology.provision.fog.smithi080:Deploy complete! 2024-03-13T18:46:29.537 DEBUG:teuthology.parallel:result is None 2024-03-13T18:46:37.532 WARNING:teuthology.provision.fog.smithi087:timed out 2024-03-13T18:46:43.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:46:43.761 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-03-13T18:46:43.761 DEBUG:teuthology.orchestra.run.smithi087:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-03-13T18:46:44.040 INFO:teuthology.provision.fog.smithi087:Node is ready 2024-03-13T18:46:44.041 DEBUG:teuthology.orchestra.run.smithi087:> hostname 2024-03-13T18:46:44.058 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087 2024-03-13T18:46:44.058 DEBUG:teuthology.orchestra.run.smithi087:> grep smithi087 /etc/hosts 2024-03-13T18:46:44.115 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087 2024-03-13T18:46:44.115 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2024-03-13T18:46:44.226 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087/smithi087/g' /etc/hosts 2024-03-13T18:46:44.253 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/172.21.15.87/172.21.15.87/g' /etc/hosts 2024-03-13T18:46:44.320 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087/smithi087/g' /etc/hostname 2024-03-13T18:46:44.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087 2024-03-13T18:46:44.453 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-03-13T18:46:44.510 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2024-03-13T18:46:44.510 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.6 (Ootpa)" 2024-03-13T18:46:44.510 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2024-03-13T18:46:44.510 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2024-03-13T18:46:44.510 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.6" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-13T18:46:44.511 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-13T18:46:44.511 INFO:teuthology.provision.fog.smithi087:Deploy complete! 2024-03-13T18:46:44.513 DEBUG:teuthology.parallel:result is None 2024-03-13T18:46:44.699 INFO:teuthology.lock.ops:Checking smithi080.front.sepia.ceph.com 2024-03-13T18:46:44.699 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-13T18:46:44.718 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com 2024-03-13T18:46:44.719 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-13T18:46:44.735 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com 2024-03-13T18:46:44.735 INFO:teuthology.lock.ops:New key found. Updating... 2024-03-13T18:46:44.753 INFO:teuthology.lock.ops:Updating [smithi080.front.sepia.ceph.com]: set os type and version on server 2024-03-13T18:46:44.753 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-13T18:46:44.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:46:44.825 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-03-13T18:46:44.964 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-13T18:46:44.964 DEBUG:teuthology.orchestra.run.smithi080:> uname -m 2024-03-13T18:46:45.022 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2024-03-13T18:46:45.022 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="Red Hat Enterprise Linux" 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="8.6 (Ootpa)" 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:ID="rhel" 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE="fedora" 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="8.6" 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:PLATFORM_ID="platform:el8" 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:ANSI_COLOR="0;31" 2024-03-13T18:46:45.080 INFO:teuthology.orchestra.run.smithi080.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-13T18:46:45.081 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.redhat.com/" 2024-03-13T18:46:45.081 INFO:teuthology.orchestra.run.smithi080.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-13T18:46:45.081 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T18:46:45.081 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-13T18:46:45.081 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T18:46:45.081 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-13T18:46:45.081 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-13T18:46:45.081 INFO:teuthology.orchestra.run.smithi080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-13T18:46:45.081 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-03-13T18:46:45.102 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server 2024-03-13T18:46:45.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-03-13T18:46:45.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:46:45.173 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-03-13T18:46:45.239 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-03-13T18:46:45.239 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-03-13T18:46:45.299 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-03-13T18:46:45.299 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-03-13T18:46:45.360 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Red Hat Enterprise Linux" 2024-03-13T18:46:45.360 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="8.6 (Ootpa)" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:ID="rhel" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="fedora" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="8.6" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el8" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.redhat.com/" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-13T18:46:45.361 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-13T18:46:45.362 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-03-13T18:46:45.394 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server 2024-03-13T18:46:45.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-03-13T18:46:45.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T18:46:45.467 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-03-13T18:46:45.604 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-03-13T18:46:45.604 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-03-13T18:46:45.662 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-03-13T18:46:45.662 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-03-13T18:46:45.719 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="Red Hat Enterprise Linux" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="8.6 (Ootpa)" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:ID="rhel" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="fedora" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="8.6" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el8" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.redhat.com/" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T18:46:45.720 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-03-13T18:46:45.721 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-13T18:46:45.721 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-13T18:46:45.721 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-13T18:46:45.721 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-13T18:46:45.721 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-03-13T18:46:45.752 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T18:46:45.858 INFO:teuthology.dispatcher.supervisor:Running job 7596979 2024-03-13T18:46:45.858 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-03-13_14:13:21-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7596979 --name yuriw-2024-03-13_14:13:21-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} -- /home/teuthworker/archive/yuriw-2024-03-13_14:13:21-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7596979/orig.config.yaml 2024-03-13T18:46:45.861 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-03-13_14:13:21-fs-wip-yuri-testing-2024-03-12-1240-reef-distro-default-smithi/7596979 2024-03-13T18:46:45.862 INFO:teuthology.dispatcher.supervisor:Job PID: 3374726 2024-03-13T18:46:45.862 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-03-13T18:48:45.966 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T18:50:46.144 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T18:52:46.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T18:54:46.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T18:54:46.597 ERROR:teuthology.dispatcher.supervisor:Child exited with code -15 2024-03-13T18:54:46.739 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi080 but it was locked by another job: /home/teuthworker/archive/yuriw-2024-03-13_14:15:30-rados-wip-yuri3-testing-2024-03-12-0851-distro-default-smithi/7597120 2024-03-13T18:54:46.739 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi087 but it was locked by another job: /home/teuthworker/archive/yuriw-2024-03-13_14:15:30-rados-wip-yuri3-testing-2024-03-12-0851-distro-default-smithi/7597122 2024-03-13T18:54:46.739 WARNING:teuthology.dispatcher.supervisor:Was going to unlock smithi106 but it was locked by another job: /home/teuthworker/archive/yuriw-2024-03-13_14:15:30-rados-wip-yuri3-testing-2024-03-12-0851-distro-default-smithi/7597122