2024-04-18T05:22:48.078 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-18T05:22:48.079 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:22:48.283 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc86a26cc10>, signals=[15]) 2024-04-18T05:22:48.284 INFO:teuthology.lock.ops:Start node 'smithi078.front.sepia.ceph.com' reimaging 2024-04-18T05:22:48.284 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T05:22:48.284 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-04-18T05:22:48.338 INFO:teuthology.lock.ops:Start node 'smithi088.front.sepia.ceph.com' reimaging 2024-04-18T05:22:48.338 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T05:22:48.338 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-04-18T05:22:48.438 INFO:teuthology.lock.ops:Start node 'smithi187.front.sepia.ceph.com' reimaging 2024-04-18T05:22:48.438 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: reset os type and version on server 2024-04-18T05:22:48.438 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-04-18T05:22:48.445 DEBUG:teuthology.provision.fog.smithi078:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-18T05:22:48.485 INFO:teuthology.provision.fog.smithi078:Scheduling deploy of rhel 8.6 2024-04-18T05:22:48.557 DEBUG:teuthology.provision.fog.smithi088:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-18T05:22:48.606 INFO:teuthology.provision.fog.smithi088:Scheduling deploy of rhel 8.6 2024-04-18T05:22:48.677 DEBUG:teuthology.provision.fog.smithi187:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-18T05:22:48.727 INFO:teuthology.provision.fog.smithi187:Scheduling deploy of rhel 8.6 2024-04-18T05:22:48.964 INFO:teuthology.orchestra.console.smithi078:Power off 2024-04-18T05:22:48.964 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T05:22:48.983 DEBUG:teuthology.orchestra.console.smithi078:power off output: Chassis Power Control: Down/Off 2024-04-18T05:22:48.984 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:49.006 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2024-04-18T05:22:49.082 INFO:teuthology.orchestra.console.smithi088:Power off 2024-04-18T05:22:49.082 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T05:22:49.103 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2024-04-18T05:22:49.103 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:49.110 INFO:teuthology.orchestra.console.smithi078:Power off completed 2024-04-18T05:22:49.124 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-04-18T05:22:49.211 INFO:teuthology.orchestra.console.smithi187:Power off 2024-04-18T05:22:49.211 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-18T05:22:49.219 INFO:teuthology.orchestra.console.smithi078:Power on 2024-04-18T05:22:49.219 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T05:22:49.226 INFO:teuthology.orchestra.console.smithi088:Power off completed 2024-04-18T05:22:49.233 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2024-04-18T05:22:49.234 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:49.242 DEBUG:teuthology.orchestra.console.smithi078:power on output: Chassis Power Control: Up/On 2024-04-18T05:22:49.242 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:49.255 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-04-18T05:22:49.265 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2024-04-18T05:22:49.327 INFO:teuthology.orchestra.console.smithi088:Power on 2024-04-18T05:22:49.327 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T05:22:49.350 DEBUG:teuthology.orchestra.console.smithi088:power on output: Chassis Power Control: Up/On 2024-04-18T05:22:49.351 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:49.358 INFO:teuthology.orchestra.console.smithi187:Power off completed 2024-04-18T05:22:49.372 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-04-18T05:22:49.460 INFO:teuthology.orchestra.console.smithi187:Power on 2024-04-18T05:22:49.460 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-18T05:22:49.481 DEBUG:teuthology.orchestra.console.smithi187:power on output: Chassis Power Control: Up/On 2024-04-18T05:22:49.482 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:49.503 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-04-18T05:22:53.270 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:53.374 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:53.382 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2024-04-18T05:22:53.482 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-04-18T05:22:53.504 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:53.613 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-04-18T05:22:57.386 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:57.483 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:57.494 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-04-18T05:22:57.591 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-18T05:22:57.596 INFO:teuthology.orchestra.console.smithi078:Power on completed 2024-04-18T05:22:57.614 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-18T05:22:57.692 INFO:teuthology.orchestra.console.smithi088:Power on completed 2024-04-18T05:22:57.697 INFO:teuthology.provision.fog.smithi078:Waiting for deploy to finish 2024-04-18T05:22:57.722 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-18T05:22:57.794 INFO:teuthology.provision.fog.smithi088:Waiting for deploy to finish 2024-04-18T05:22:57.823 INFO:teuthology.orchestra.console.smithi187:Power on completed 2024-04-18T05:22:57.924 INFO:teuthology.provision.fog.smithi187:Waiting for deploy to finish 2024-04-18T05:26:15.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:18.206 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:26:24.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:27.262 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:26:30.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:33.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:33.310 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-18T05:26:36.318 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:26:39.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:42.322 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:42.366 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-18T05:26:45.374 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:26:45.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:48.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:48.446 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-04-18T05:26:51.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:51.422 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-18T05:26:54.430 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:26:54.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:57.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:26:57.502 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-04-18T05:27:00.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:00.478 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-18T05:27:03.486 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:27:03.504 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:06.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:06.558 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-04-18T05:27:09.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:09.534 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-18T05:27:12.542 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:27:12.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:15.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:15.618 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-04-18T05:27:18.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:18.590 WARNING:teuthology.provision.fog.smithi187:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-04-18T05:27:21.598 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:27:21.619 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:24.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:24.670 WARNING:teuthology.provision.fog.smithi078:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-04-18T05:27:27.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:30.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:45.955 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:27:51.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:27:55.038 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:28:01.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:28:04.094 WARNING:teuthology.provision.fog.smithi088:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-04-18T05:28:10.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:28:10.409 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T05:28:10.410 DEBUG:teuthology.orchestra.run.smithi088:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T05:28:24.595 WARNING:teuthology.provision.fog.smithi187:timed out 2024-04-18T05:28:30.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:28:30.682 WARNING:teuthology.provision.fog.smithi078:timed out 2024-04-18T05:28:30.760 INFO:teuthology.provision.fog.smithi088:Node is ready 2024-04-18T05:28:30.760 DEBUG:teuthology.orchestra.run.smithi088:> hostname 2024-04-18T05:28:30.821 INFO:teuthology.orchestra.run.smithi088.stdout:smithi088 2024-04-18T05:28:30.821 DEBUG:teuthology.orchestra.run.smithi088:> grep smithi088 /etc/hosts 2024-04-18T05:28:30.839 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T05:28:30.839 DEBUG:teuthology.orchestra.run.smithi187:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T05:28:30.879 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88 smithi088.front.sepia.ceph.com smithi088 2024-04-18T05:28:30.880 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088 2024-04-18T05:28:31.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088/smithi088/g' /etc/hosts 2024-04-18T05:28:31.051 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/172.21.15.88/172.21.15.88/g' /etc/hosts 2024-04-18T05:28:31.121 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i -e 's/smithi088/smithi088/g' /etc/hostname 2024-04-18T05:28:31.132 INFO:teuthology.provision.fog.smithi187:Node is ready 2024-04-18T05:28:31.132 DEBUG:teuthology.orchestra.run.smithi187:> hostname 2024-04-18T05:28:31.149 INFO:teuthology.orchestra.run.smithi187.stdout:smithi187 2024-04-18T05:28:31.149 DEBUG:teuthology.orchestra.run.smithi187:> grep smithi187 /etc/hosts 2024-04-18T05:28:31.189 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname smithi088 2024-04-18T05:28:31.203 INFO:teuthology.orchestra.run.smithi187.stdout:172.21.15.187 smithi187.front.sepia.ceph.com smithi187 2024-04-18T05:28:31.203 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname smithi187 2024-04-18T05:28:31.257 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:ID="rhel" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="fedora" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="8.6" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T05:28:31.315 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T05:28:31.316 INFO:teuthology.orchestra.run.smithi088.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T05:28:31.316 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:28:31.316 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-18T05:28:31.316 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:28:31.316 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T05:28:31.316 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T05:28:31.316 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T05:28:31.317 INFO:teuthology.provision.fog.smithi088:Deploy complete! 2024-04-18T05:28:31.318 DEBUG:teuthology.parallel:result is None 2024-04-18T05:28:31.324 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/smithi187/smithi187/g' /etc/hosts 2024-04-18T05:28:31.351 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/172.21.15.187/172.21.15.187/g' /etc/hosts 2024-04-18T05:28:31.417 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i -e 's/smithi187/smithi187/g' /etc/hostname 2024-04-18T05:28:31.481 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname smithi187 2024-04-18T05:28:31.545 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.6" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:28:31.601 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-18T05:28:31.602 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:28:31.602 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T05:28:31.602 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T05:28:31.602 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T05:28:31.602 INFO:teuthology.provision.fog.smithi187:Deploy complete! 2024-04-18T05:28:31.602 DEBUG:teuthology.parallel:result is None 2024-04-18T05:28:36.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:28:36.929 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-18T05:28:36.929 DEBUG:teuthology.orchestra.run.smithi078:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-18T05:28:37.197 INFO:teuthology.provision.fog.smithi078:Node is ready 2024-04-18T05:28:37.198 DEBUG:teuthology.orchestra.run.smithi078:> hostname 2024-04-18T05:28:37.258 INFO:teuthology.orchestra.run.smithi078.stdout:smithi078 2024-04-18T05:28:37.258 DEBUG:teuthology.orchestra.run.smithi078:> grep smithi078 /etc/hosts 2024-04-18T05:28:37.318 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78 smithi078.front.sepia.ceph.com smithi078 2024-04-18T05:28:37.318 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname smithi078 2024-04-18T05:28:37.430 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/smithi078/smithi078/g' /etc/hosts 2024-04-18T05:28:37.463 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/172.21.15.78/172.21.15.78/g' /etc/hosts 2024-04-18T05:28:37.538 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i -e 's/smithi078/smithi078/g' /etc/hostname 2024-04-18T05:28:37.612 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname smithi078 2024-04-18T05:28:37.684 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-04-18T05:28:37.785 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T05:28:37.785 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T05:28:37.785 INFO:teuthology.orchestra.run.smithi078.stdout:ID="rhel" 2024-04-18T05:28:37.785 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="fedora" 2024-04-18T05:28:37.785 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8.6" 2024-04-18T05:28:37.785 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T05:28:37.786 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T05:28:37.787 INFO:teuthology.provision.fog.smithi078:Deploy complete! 2024-04-18T05:28:37.787 DEBUG:teuthology.parallel:result is None 2024-04-18T05:28:38.019 INFO:teuthology.lock.ops:Checking smithi078.front.sepia.ceph.com 2024-04-18T05:28:38.020 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T05:28:38.054 INFO:teuthology.lock.ops:Checking smithi088.front.sepia.ceph.com 2024-04-18T05:28:38.054 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T05:28:38.070 INFO:teuthology.lock.ops:Checking smithi187.front.sepia.ceph.com 2024-04-18T05:28:38.070 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-18T05:28:38.088 INFO:teuthology.lock.ops:Updating [smithi078.front.sepia.ceph.com]: set os type and version on server 2024-04-18T05:28:38.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-18T05:28:38.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:28:38.159 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-04-18T05:28:38.230 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-18T05:28:38.231 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-04-18T05:28:38.290 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-04-18T05:28:38.291 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:ID="rhel" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="fedora" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8.6" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:28:38.351 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-18T05:28:38.352 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:28:38.352 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T05:28:38.352 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T05:28:38.352 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T05:28:38.352 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-04-18T05:28:38.370 INFO:teuthology.lock.ops:Updating [smithi088.front.sepia.ceph.com]: set os type and version on server 2024-04-18T05:28:38.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-18T05:28:38.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:28:38.441 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-04-18T05:28:38.511 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-18T05:28:38.511 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-04-18T05:28:38.572 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-04-18T05:28:38.572 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-04-18T05:28:38.632 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T05:28:38.632 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T05:28:38.632 INFO:teuthology.orchestra.run.smithi088.stdout:ID="rhel" 2024-04-18T05:28:38.632 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="fedora" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="8.6" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T05:28:38.633 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T05:28:38.634 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-04-18T05:28:38.653 INFO:teuthology.lock.ops:Updating [smithi187.front.sepia.ceph.com]: set os type and version on server 2024-04-18T05:28:38.654 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-18T05:28:38.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T05:28:38.727 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-18T05:28:38.799 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-18T05:28:38.799 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-04-18T05:28:38.858 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-04-18T05:28:38.858 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-04-18T05:28:38.917 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux" 2024-04-18T05:28:38.917 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.6 (Ootpa)" 2024-04-18T05:28:38.917 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel" 2024-04-18T05:28:38.917 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora" 2024-04-18T05:28:38.917 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.6" 2024-04-18T05:28:38.917 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-18T05:28:38.918 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-18T05:28:38.919 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-04-18T05:28:38.937 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:28:39.044 INFO:teuthology.dispatcher.supervisor:Running job 7661590 2024-04-18T05:28:39.044 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661590 --name teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/pjd}} -- /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661590/orig.config.yaml 2024-04-18T05:28:39.047 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-17_22:24:03-fs-reef-distro-default-smithi/7661590 2024-04-18T05:28:39.048 INFO:teuthology.dispatcher.supervisor:Job PID: 1597748 2024-04-18T05:28:39.048 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-18T05:30:39.147 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:32:39.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:34:39.483 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:36:39.641 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:38:39.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:40:39.988 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:42:40.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:44:40.276 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:46:40.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:48:40.647 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T05:48:40.712 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1