2024-04-13T18:22:21.642 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-13T18:22:21.643 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:22:21.819 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f42002cac10>, signals=[15]) 2024-04-13T18:22:21.819 INFO:teuthology.lock.ops:Start node 'smithi016.front.sepia.ceph.com' reimaging 2024-04-13T18:22:21.820 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: reset os type and version on server 2024-04-13T18:22:21.820 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-04-13T18:22:21.845 INFO:teuthology.lock.ops:Start node 'smithi045.front.sepia.ceph.com' reimaging 2024-04-13T18:22:21.845 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: reset os type and version on server 2024-04-13T18:22:21.846 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-04-13T18:22:21.869 INFO:teuthology.lock.ops:Start node 'smithi172.front.sepia.ceph.com' reimaging 2024-04-13T18:22:21.869 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: reset os type and version on server 2024-04-13T18:22:21.869 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-04-13T18:22:21.976 DEBUG:teuthology.provision.fog.smithi016:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-13T18:22:21.987 DEBUG:teuthology.provision.fog.smithi045:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-13T18:22:22.009 DEBUG:teuthology.provision.fog.smithi172:Requesting image smithi_rhel_8.6 (ID 5) 2024-04-13T18:22:22.029 INFO:teuthology.provision.fog.smithi016:Scheduling deploy of rhel 8.6 2024-04-13T18:22:22.034 INFO:teuthology.provision.fog.smithi045:Scheduling deploy of rhel 8.6 2024-04-13T18:22:22.048 INFO:teuthology.provision.fog.smithi172:Scheduling deploy of rhel 8.6 2024-04-13T18:22:22.325 INFO:teuthology.orchestra.console.smithi045:Power off 2024-04-13T18:22:22.325 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T18:22:22.345 INFO:teuthology.orchestra.console.smithi172:Power off 2024-04-13T18:22:22.345 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T18:22:22.360 INFO:teuthology.orchestra.console.smithi016:Power off 2024-04-13T18:22:22.360 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-13T18:22:22.369 DEBUG:teuthology.orchestra.console.smithi172:power off output: Chassis Power Control: Down/Off 2024-04-13T18:22:22.369 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:22.379 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2024-04-13T18:22:22.379 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:22.390 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2024-04-13T18:22:22.392 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:22.403 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2024-04-13T18:22:22.404 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-04-13T18:22:22.415 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2024-04-13T18:22:22.505 INFO:teuthology.orchestra.console.smithi172:Power off completed 2024-04-13T18:22:22.505 INFO:teuthology.orchestra.console.smithi045:Power off completed 2024-04-13T18:22:22.516 INFO:teuthology.orchestra.console.smithi016:Power off completed 2024-04-13T18:22:22.606 INFO:teuthology.orchestra.console.smithi172:Power on 2024-04-13T18:22:22.606 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-13T18:22:22.615 INFO:teuthology.orchestra.console.smithi045:Power on 2024-04-13T18:22:22.615 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-13T18:22:22.622 INFO:teuthology.orchestra.console.smithi016:Power on 2024-04-13T18:22:22.623 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-13T18:22:22.633 DEBUG:teuthology.orchestra.console.smithi172:power on output: Chassis Power Control: Up/On 2024-04-13T18:22:22.634 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:22.643 DEBUG:teuthology.orchestra.console.smithi016:power on output: Chassis Power Control: Up/On 2024-04-13T18:22:22.644 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:22.652 DEBUG:teuthology.orchestra.console.smithi045:power on output: Chassis Power Control: Up/On 2024-04-13T18:22:22.652 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:22.661 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is off 2024-04-13T18:22:22.670 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2024-04-13T18:22:22.678 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-04-13T18:22:26.662 DEBUG:teuthology.orchestra.console.smithi172:pexpect command: ipmitool -H smithi172.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:26.673 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:26.682 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:26.776 DEBUG:teuthology.orchestra.console.smithi172:check power output: Chassis Power is on 2024-04-13T18:22:26.784 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2024-04-13T18:22:26.793 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-04-13T18:22:26.876 INFO:teuthology.orchestra.console.smithi172:Power on completed 2024-04-13T18:22:26.977 INFO:teuthology.provision.fog.smithi172:Waiting for deploy to finish 2024-04-13T18:22:30.786 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:30.793 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-13T18:22:30.896 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2024-04-13T18:22:30.902 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-04-13T18:22:30.998 INFO:teuthology.orchestra.console.smithi016:Power on completed 2024-04-13T18:22:31.003 INFO:teuthology.orchestra.console.smithi045:Power on completed 2024-04-13T18:22:31.099 INFO:teuthology.provision.fog.smithi016:Waiting for deploy to finish 2024-04-13T18:22:31.104 INFO:teuthology.provision.fog.smithi045:Waiting for deploy to finish 2024-04-13T18:25:02.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:05.886 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-13T18:25:11.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:14.942 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-13T18:25:17.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:20.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:20.990 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-04-13T18:25:24.002 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-13T18:25:26.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:30.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:30.046 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-04-13T18:25:33.090 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-13T18:25:36.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:39.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:39.102 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-04-13T18:25:42.174 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-13T18:25:45.105 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:48.158 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-04-13T18:25:48.176 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:51.230 WARNING:teuthology.provision.fog.smithi045:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-13T18:25:54.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:57.217 WARNING:teuthology.provision.fog.smithi016:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-04-13T18:25:57.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:25:59.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:26:02.206 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:26:03.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:26:08.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:26:11.262 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:26:17.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:26:20.318 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:26:26.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:26:29.374 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:26:35.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:26:38.431 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:26:44.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:26:47.518 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:26:53.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:26:56.574 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:26:57.234 WARNING:teuthology.provision.fog.smithi045:timed out 2024-04-13T18:27:02.579 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:27:03.223 WARNING:teuthology.provision.fog.smithi016:timed out 2024-04-13T18:27:03.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:27:03.498 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-13T18:27:03.499 DEBUG:teuthology.orchestra.run.smithi045:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-13T18:27:03.803 INFO:teuthology.provision.fog.smithi045:Node is ready 2024-04-13T18:27:03.803 DEBUG:teuthology.orchestra.run.smithi045:> hostname 2024-04-13T18:27:03.824 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045.front.sepia.ceph.com 2024-04-13T18:27:03.824 DEBUG:teuthology.orchestra.run.smithi045:> grep smithi045.front.sepia.ceph.com /etc/hosts 2024-04-13T18:27:03.886 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45 smithi045.front.sepia.ceph.com smithi045 2024-04-13T18:27:03.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname smithi045 2024-04-13T18:27:04.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/smithi045.front.sepia.ceph.com/smithi045/g' /etc/hosts 2024-04-13T18:27:04.035 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/172.21.15.45/172.21.15.45/g' /etc/hosts 2024-04-13T18:27:04.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i -e 's/smithi045.front.sepia.ceph.com/smithi045/g' /etc/hostname 2024-04-13T18:27:04.184 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname smithi045 2024-04-13T18:27:04.258 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-04-13T18:27:04.319 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:27:04.319 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:27:04.319 INFO:teuthology.orchestra.run.smithi045.stdout:ID="rhel" 2024-04-13T18:27:04.319 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="fedora" 2024-04-13T18:27:04.319 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8.6" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:27:04.320 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:27:04.321 INFO:teuthology.provision.fog.smithi045:Deploy complete! 2024-04-13T18:27:04.322 DEBUG:teuthology.parallel:result is None 2024-04-13T18:27:09.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:27:09.459 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-13T18:27:09.460 DEBUG:teuthology.orchestra.run.smithi016:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-13T18:27:09.728 INFO:teuthology.provision.fog.smithi016:Node is ready 2024-04-13T18:27:09.729 DEBUG:teuthology.orchestra.run.smithi016:> hostname 2024-04-13T18:27:09.749 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016.front.sepia.ceph.com 2024-04-13T18:27:09.750 DEBUG:teuthology.orchestra.run.smithi016:> grep smithi016.front.sepia.ceph.com /etc/hosts 2024-04-13T18:27:09.811 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16 smithi016.front.sepia.ceph.com smithi016 2024-04-13T18:27:09.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2024-04-13T18:27:09.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hosts 2024-04-13T18:27:09.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/172.21.15.16/172.21.15.16/g' /etc/hosts 2024-04-13T18:27:10.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i -e 's/smithi016.front.sepia.ceph.com/smithi016/g' /etc/hostname 2024-04-13T18:27:10.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname smithi016 2024-04-13T18:27:10.179 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-04-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:27:10.240 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:27:10.241 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:27:10.242 INFO:teuthology.provision.fog.smithi016:Deploy complete! 2024-04-13T18:27:10.242 DEBUG:teuthology.parallel:result is None 2024-04-13T18:27:20.991 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:27:26.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:27:30.050 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:27:36.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:27:39.134 WARNING:teuthology.provision.fog.smithi172:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-13T18:27:45.139 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:27:45.343 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-13T18:27:45.344 DEBUG:teuthology.orchestra.run.smithi172:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-13T18:28:05.603 INFO:teuthology.provision.fog.smithi172:Node is ready 2024-04-13T18:28:05.604 DEBUG:teuthology.orchestra.run.smithi172:> hostname 2024-04-13T18:28:05.626 INFO:teuthology.orchestra.run.smithi172.stdout:smithi172.front.sepia.ceph.com 2024-04-13T18:28:05.626 DEBUG:teuthology.orchestra.run.smithi172:> grep smithi172.front.sepia.ceph.com /etc/hosts 2024-04-13T18:28:05.685 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.15.172 smithi172.front.sepia.ceph.com smithi172 2024-04-13T18:28:05.685 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172 2024-04-13T18:28:05.820 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi172.front.sepia.ceph.com/smithi172/g' /etc/hosts 2024-04-13T18:28:05.852 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/172.21.15.172/172.21.15.172/g' /etc/hosts 2024-04-13T18:28:05.921 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i -e 's/smithi172.front.sepia.ceph.com/smithi172/g' /etc/hostname 2024-04-13T18:28:05.989 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname smithi172 2024-04-13T18:28:06.056 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-04-13T18:28:06.113 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:28:06.113 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:28:06.113 INFO:teuthology.orchestra.run.smithi172.stdout:ID="rhel" 2024-04-13T18:28:06.113 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="fedora" 2024-04-13T18:28:06.113 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8.6" 2024-04-13T18:28:06.113 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:28:06.113 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:28:06.113 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:28:06.114 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:28:06.114 INFO:teuthology.provision.fog.smithi172:Deploy complete! 2024-04-13T18:28:06.115 DEBUG:teuthology.parallel:result is None 2024-04-13T18:28:06.312 INFO:teuthology.lock.ops:Checking smithi016.front.sepia.ceph.com 2024-04-13T18:28:06.312 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-13T18:28:06.340 INFO:teuthology.lock.ops:Checking smithi045.front.sepia.ceph.com 2024-04-13T18:28:06.341 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-13T18:28:06.357 INFO:teuthology.lock.ops:Checking smithi172.front.sepia.ceph.com 2024-04-13T18:28:06.357 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-13T18:28:06.384 INFO:teuthology.lock.ops:Updating [smithi016.front.sepia.ceph.com]: set os type and version on server 2024-04-13T18:28:06.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-13T18:28:06.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:28:06.472 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-04-13T18:28:06.616 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-04-13T18:28:06.616 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-04-13T18:28:06.677 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-04-13T18:28:06.677 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-04-13T18:28:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:28:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:28:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2024-04-13T18:28:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2024-04-13T18:28:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2024-04-13T18:28:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:28:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:28:06.738 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:28:06.739 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:28:06.740 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-04-13T18:28:06.757 INFO:teuthology.lock.ops:Updating [smithi045.front.sepia.ceph.com]: set os type and version on server 2024-04-13T18:28:06.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-13T18:28:06.758 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:28:06.833 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-13T18:28:06.978 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-13T18:28:06.978 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-04-13T18:28:07.039 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-04-13T18:28:07.039 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:ID="rhel" 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="fedora" 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="8.6" 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-04-13T18:28:07.100 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:28:07.101 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:28:07.101 INFO:teuthology.orchestra.run.smithi045.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:28:07.101 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:28:07.101 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-13T18:28:07.101 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:28:07.101 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:28:07.101 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:28:07.101 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:28:07.101 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-04-13T18:28:07.123 INFO:teuthology.lock.ops:Updating [smithi172.front.sepia.ceph.com]: set os type and version on server 2024-04-13T18:28:07.124 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-13T18:28:07.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-13T18:28:07.193 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-04-13T18:28:07.260 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-13T18:28:07.261 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-04-13T18:28:07.318 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-04-13T18:28:07.319 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-04-13T18:28:07.376 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="Red Hat Enterprise Linux" 2024-04-13T18:28:07.376 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="8.6 (Ootpa)" 2024-04-13T18:28:07.377 INFO:teuthology.orchestra.run.smithi172.stdout:ID="rhel" 2024-04-13T18:28:07.377 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="fedora" 2024-04-13T18:28:07.377 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="8.6" 2024-04-13T18:28:07.377 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el8" 2024-04-13T18:28:07.377 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-13T18:28:07.377 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-04-13T18:28:07.377 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-13T18:28:07.377 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://www.redhat.com/" 2024-04-13T18:28:07.378 INFO:teuthology.orchestra.run.smithi172.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-13T18:28:07.378 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-13T18:28:07.378 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-13T18:28:07.378 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-13T18:28:07.378 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-13T18:28:07.378 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-13T18:28:07.378 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-13T18:28:07.379 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-04-13T18:28:07.399 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:28:07.495 INFO:teuthology.dispatcher.supervisor:Running job 7651581 2024-04-13T18:28:07.495 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-10_22:24:02-fs-reef-distro-default-smithi/7651581 --name teuthology-2024-04-10_22:24:02-fs-reef-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/fsstress}} -- /home/teuthworker/archive/teuthology-2024-04-10_22:24:02-fs-reef-distro-default-smithi/7651581/orig.config.yaml 2024-04-13T18:28:07.498 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-10_22:24:02-fs-reef-distro-default-smithi/7651581 2024-04-13T18:28:07.499 INFO:teuthology.dispatcher.supervisor:Job PID: 1825698 2024-04-13T18:28:07.499 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-13T18:30:07.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:32:07.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:34:07.937 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:36:08.119 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:38:08.303 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:40:08.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:42:08.651 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:44:08.827 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:46:09.015 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:48:09.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:50:09.355 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-13T18:50:09.432 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1