2024-02-24T10:12:49.791 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-24T10:12:49.791 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:12:50.352 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6d1ea74c10>, signals=[15]) 2024-02-24T10:12:50.353 INFO:teuthology.lock.ops:Start node 'smithi103.front.sepia.ceph.com' reimaging 2024-02-24T10:12:50.353 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: reset os type and version on server 2024-02-24T10:12:50.353 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-02-24T10:12:52.125 INFO:teuthology.lock.ops:Start node 'smithi146.front.sepia.ceph.com' reimaging 2024-02-24T10:12:52.125 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: reset os type and version on server 2024-02-24T10:12:52.125 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-02-24T10:12:52.141 INFO:teuthology.lock.ops:Start node 'smithi158.front.sepia.ceph.com' reimaging 2024-02-24T10:12:52.141 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: reset os type and version on server 2024-02-24T10:12:52.141 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2024-02-24T10:12:52.440 DEBUG:teuthology.provision.fog.smithi103:Requesting image smithi_centos_9.stream (ID 10) 2024-02-24T10:12:52.467 DEBUG:teuthology.provision.fog.smithi146:Requesting image smithi_centos_9.stream (ID 10) 2024-02-24T10:12:52.514 DEBUG:teuthology.provision.fog.smithi158:Requesting image smithi_centos_9.stream (ID 10) 2024-02-24T10:12:52.622 INFO:teuthology.provision.fog.smithi103:Scheduling deploy of centos 9.stream 2024-02-24T10:12:52.684 INFO:teuthology.provision.fog.smithi146:Scheduling deploy of centos 9.stream 2024-02-24T10:12:52.770 INFO:teuthology.provision.fog.smithi158:Scheduling deploy of centos 9.stream 2024-02-24T10:12:53.745 INFO:teuthology.orchestra.console.smithi103:Power off 2024-02-24T10:12:53.745 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-24T10:12:53.768 DEBUG:teuthology.orchestra.console.smithi103:power off output: Chassis Power Control: Down/Off 2024-02-24T10:12:53.768 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:12:53.794 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2024-02-24T10:12:53.806 INFO:teuthology.orchestra.console.smithi146:Power off 2024-02-24T10:12:53.806 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-24T10:12:53.825 DEBUG:teuthology.orchestra.console.smithi146:power off output: Chassis Power Control: Down/Off 2024-02-24T10:12:53.826 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:12:53.850 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2024-02-24T10:12:53.890 INFO:teuthology.orchestra.console.smithi158:Power off 2024-02-24T10:12:53.890 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-24T10:12:53.913 DEBUG:teuthology.orchestra.console.smithi158:power off output: Chassis Power Control: Down/Off 2024-02-24T10:12:53.913 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:12:53.934 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2024-02-24T10:12:57.799 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:12:57.851 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:12:57.908 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2024-02-24T10:12:57.935 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:12:57.961 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2024-02-24T10:12:58.043 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2024-02-24T10:13:01.913 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:01.964 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:02.023 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2024-02-24T10:13:02.043 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:02.072 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2024-02-24T10:13:02.151 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2024-02-24T10:13:02.173 INFO:teuthology.orchestra.console.smithi146:Power off completed 2024-02-24T10:13:02.252 INFO:teuthology.orchestra.console.smithi158:Power off completed 2024-02-24T10:13:02.274 INFO:teuthology.orchestra.console.smithi146:Power on 2024-02-24T10:13:02.275 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-24T10:13:02.297 DEBUG:teuthology.orchestra.console.smithi146:power on output: Chassis Power Control: Up/On 2024-02-24T10:13:02.297 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:02.317 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2024-02-24T10:13:02.353 INFO:teuthology.orchestra.console.smithi158:Power on 2024-02-24T10:13:02.353 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-24T10:13:02.374 DEBUG:teuthology.orchestra.console.smithi158:power on output: Chassis Power Control: Up/On 2024-02-24T10:13:02.374 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:02.394 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2024-02-24T10:13:06.027 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:06.136 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2024-02-24T10:13:06.237 INFO:teuthology.orchestra.console.smithi103:Power off completed 2024-02-24T10:13:06.318 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:06.338 INFO:teuthology.orchestra.console.smithi103:Power on 2024-02-24T10:13:06.338 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-24T10:13:06.357 DEBUG:teuthology.orchestra.console.smithi103:power on output: Chassis Power Control: Up/On 2024-02-24T10:13:06.358 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:06.377 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2024-02-24T10:13:06.395 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:06.426 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is off 2024-02-24T10:13:06.502 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is off 2024-02-24T10:13:10.381 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:10.427 DEBUG:teuthology.orchestra.console.smithi146:pexpect command: ipmitool -H smithi146.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:10.490 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is off 2024-02-24T10:13:10.503 DEBUG:teuthology.orchestra.console.smithi158:pexpect command: ipmitool -H smithi158.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:10.535 DEBUG:teuthology.orchestra.console.smithi146:check power output: Chassis Power is on 2024-02-24T10:13:10.636 INFO:teuthology.orchestra.console.smithi146:Power on completed 2024-02-24T10:13:10.728 DEBUG:teuthology.orchestra.console.smithi158:check power output: Chassis Power is on 2024-02-24T10:13:10.737 INFO:teuthology.provision.fog.smithi146:Waiting for deploy to finish 2024-02-24T10:13:10.829 INFO:teuthology.orchestra.console.smithi158:Power on completed 2024-02-24T10:13:10.929 INFO:teuthology.provision.fog.smithi158:Waiting for deploy to finish 2024-02-24T10:13:14.494 DEBUG:teuthology.orchestra.console.smithi103:pexpect command: ipmitool -H smithi103.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-24T10:13:14.602 DEBUG:teuthology.orchestra.console.smithi103:check power output: Chassis Power is on 2024-02-24T10:13:14.703 INFO:teuthology.orchestra.console.smithi103:Power on completed 2024-02-24T10:13:14.804 INFO:teuthology.provision.fog.smithi103:Waiting for deploy to finish 2024-02-24T10:17:34.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:17:37.500 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-02-24T10:17:43.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:17:46.556 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-02-24T10:17:52.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:17:55.612 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-02-24T10:18:01.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:04.668 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-02-24T10:18:10.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:13.756 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-02-24T10:18:19.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:20.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:22.812 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-02-24T10:18:23.964 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:18:28.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:29.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:31.900 WARNING:teuthology.provision.fog.smithi146:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-02-24T10:18:33.020 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:18:37.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:39.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:42.076 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:18:48.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:18:51.132 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:18:57.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:00.220 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:19:01.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:04.509 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-02-24T10:19:06.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:09.276 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:19:10.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:13.568 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-02-24T10:19:15.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:18.332 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:19:19.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:22.652 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-02-24T10:19:24.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:27.388 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:19:28.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:31.708 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-02-24T10:19:33.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:36.444 WARNING:teuthology.provision.fog.smithi158:[Errno None] Unable to connect to port 22 on 172.21.15.158 2024-02-24T10:19:37.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:37.908 WARNING:teuthology.provision.fog.smithi146:timed out 2024-02-24T10:19:40.764 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-02-24T10:19:42.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:43.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:44.162 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-24T10:19:44.163 DEBUG:teuthology.orchestra.run.smithi146:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-24T10:19:44.486 INFO:teuthology.provision.fog.smithi146:Node is ready 2024-02-24T10:19:44.486 DEBUG:teuthology.orchestra.run.smithi146:> hostname 2024-02-24T10:19:44.501 INFO:teuthology.orchestra.run.smithi146.stdout:smithi146.front.sepia.ceph.com 2024-02-24T10:19:44.501 DEBUG:teuthology.orchestra.run.smithi146:> grep smithi146.front.sepia.ceph.com /etc/hosts 2024-02-24T10:19:44.556 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.15.146 smithi146.front.sepia.ceph.com smithi146 2024-02-24T10:19:44.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146 2024-02-24T10:19:44.656 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hosts 2024-02-24T10:19:44.683 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/172.21.15.146/172.21.15.146/g' /etc/hosts 2024-02-24T10:19:44.751 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i -e 's/smithi146.front.sepia.ceph.com/smithi146/g' /etc/hostname 2024-02-24T10:19:44.818 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname smithi146 2024-02-24T10:19:44.884 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-02-24T10:19:44.939 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-02-24T10:19:44.939 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-02-24T10:19:44.939 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-24T10:19:44.940 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-24T10:19:44.941 INFO:teuthology.provision.fog.smithi146:Deploy complete! 2024-02-24T10:19:44.942 DEBUG:teuthology.parallel:result is None 2024-02-24T10:19:46.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:49.884 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-02-24T10:19:55.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:19:58.972 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-02-24T10:20:04.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:20:08.060 WARNING:teuthology.provision.fog.smithi103:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-02-24T10:20:14.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:20:42.468 WARNING:teuthology.provision.fog.smithi158:timed out 2024-02-24T10:20:48.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:20:48.827 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-24T10:20:48.827 DEBUG:teuthology.orchestra.run.smithi158:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-24T10:20:49.140 INFO:teuthology.provision.fog.smithi158:Node is ready 2024-02-24T10:20:49.140 DEBUG:teuthology.orchestra.run.smithi158:> hostname 2024-02-24T10:20:49.154 INFO:teuthology.orchestra.run.smithi158.stdout:smithi158.front.sepia.ceph.com 2024-02-24T10:20:49.154 DEBUG:teuthology.orchestra.run.smithi158:> grep smithi158.front.sepia.ceph.com /etc/hosts 2024-02-24T10:20:49.213 INFO:teuthology.orchestra.run.smithi158.stdout:172.21.15.158 smithi158.front.sepia.ceph.com smithi158 2024-02-24T10:20:49.213 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158 2024-02-24T10:20:49.329 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hosts 2024-02-24T10:20:49.356 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/172.21.15.158/172.21.15.158/g' /etc/hosts 2024-02-24T10:20:49.426 DEBUG:teuthology.orchestra.run.smithi158:> sudo sed -i -e 's/smithi158.front.sepia.ceph.com/smithi158/g' /etc/hostname 2024-02-24T10:20:49.497 DEBUG:teuthology.orchestra.run.smithi158:> sudo hostname smithi158 2024-02-24T10:20:49.567 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2024-02-24T10:20:49.625 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-24T10:20:49.626 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-24T10:20:49.627 INFO:teuthology.provision.fog.smithi158:Deploy complete! 2024-02-24T10:20:49.627 DEBUG:teuthology.parallel:result is None 2024-02-24T10:21:14.093 WARNING:teuthology.provision.fog.smithi103:timed out 2024-02-24T10:21:20.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:21:20.420 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-24T10:21:20.420 DEBUG:teuthology.orchestra.run.smithi103:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-24T10:21:20.747 INFO:teuthology.provision.fog.smithi103:Node is ready 2024-02-24T10:21:20.747 DEBUG:teuthology.orchestra.run.smithi103:> hostname 2024-02-24T10:21:20.762 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103.front.sepia.ceph.com 2024-02-24T10:21:20.762 DEBUG:teuthology.orchestra.run.smithi103:> grep smithi103.front.sepia.ceph.com /etc/hosts 2024-02-24T10:21:20.819 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103 smithi103.front.sepia.ceph.com smithi103 2024-02-24T10:21:20.819 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname smithi103 2024-02-24T10:21:20.929 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/smithi103.front.sepia.ceph.com/smithi103/g' /etc/hosts 2024-02-24T10:21:20.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/172.21.15.103/172.21.15.103/g' /etc/hosts 2024-02-24T10:21:21.026 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i -e 's/smithi103.front.sepia.ceph.com/smithi103/g' /etc/hostname 2024-02-24T10:21:21.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname smithi103 2024-02-24T10:21:21.161 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-02-24T10:21:21.215 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-02-24T10:21:21.215 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-02-24T10:21:21.215 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-02-24T10:21:21.215 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-02-24T10:21:21.215 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-24T10:21:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-24T10:21:21.216 INFO:teuthology.provision.fog.smithi103:Deploy complete! 2024-02-24T10:21:21.217 DEBUG:teuthology.parallel:result is None 2024-02-24T10:21:21.427 INFO:teuthology.lock.ops:Checking smithi103.front.sepia.ceph.com 2024-02-24T10:21:21.427 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-24T10:21:21.448 INFO:teuthology.lock.ops:Checking smithi146.front.sepia.ceph.com 2024-02-24T10:21:21.448 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-24T10:21:21.465 INFO:teuthology.lock.ops:Checking smithi158.front.sepia.ceph.com 2024-02-24T10:21:21.465 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-24T10:21:21.482 INFO:teuthology.lock.ops:Updating [smithi103.front.sepia.ceph.com]: set os type and version on server 2024-02-24T10:21:21.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-02-24T10:21:21.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:21:21.591 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-02-24T10:21:21.654 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-02-24T10:21:21.654 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-02-24T10:21:21.709 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-02-24T10:21:21.709 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-02-24T10:21:21.763 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-02-24T10:21:21.763 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-02-24T10:21:21.763 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-02-24T10:21:21.763 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-02-24T10:21:21.763 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-02-24T10:21:21.763 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-02-24T10:21:21.763 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-24T10:21:21.763 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-02-24T10:21:21.764 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-02-24T10:21:21.764 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-24T10:21:21.764 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-02-24T10:21:21.764 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-24T10:21:21.764 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-24T10:21:21.764 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-24T10:21:21.764 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-02-24T10:21:21.790 INFO:teuthology.lock.ops:Updating [smithi146.front.sepia.ceph.com]: set os type and version on server 2024-02-24T10:21:21.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-02-24T10:21:21.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:21:21.878 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-02-24T10:21:22.046 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-02-24T10:21:22.046 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-02-24T10:21:22.100 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-02-24T10:21:22.101 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-02-24T10:21:22.155 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-24T10:21:22.156 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-02-24T10:21:22.156 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-24T10:21:22.156 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-24T10:21:22.156 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-24T10:21:22.156 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-02-24T10:21:22.173 INFO:teuthology.lock.ops:Updating [smithi158.front.sepia.ceph.com]: set os type and version on server 2024-02-24T10:21:22.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi158.front.sepia.ceph.com' 2024-02-24T10:21:22.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi158.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-24T10:21:22.261 DEBUG:teuthology.orchestra.run.smithi158:> true 2024-02-24T10:21:22.430 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi158.front.sepia.ceph.com' 2024-02-24T10:21:22.430 DEBUG:teuthology.orchestra.run.smithi158:> uname -m 2024-02-24T10:21:22.484 INFO:teuthology.orchestra.run.smithi158.stdout:x86_64 2024-02-24T10:21:22.484 DEBUG:teuthology.orchestra.run.smithi158:> cat /etc/os-release 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:NAME="CentOS Stream" 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION="9" 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:ID="centos" 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:ID_LIKE="rhel fedora" 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:VERSION_ID="9" 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:PLATFORM_ID="platform:el9" 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:ANSI_COLOR="0;31" 2024-02-24T10:21:22.538 INFO:teuthology.orchestra.run.smithi158.stdout:LOGO="fedora-logo-icon" 2024-02-24T10:21:22.539 INFO:teuthology.orchestra.run.smithi158.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-24T10:21:22.539 INFO:teuthology.orchestra.run.smithi158.stdout:HOME_URL="https://centos.org/" 2024-02-24T10:21:22.539 INFO:teuthology.orchestra.run.smithi158.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-24T10:21:22.539 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-24T10:21:22.539 INFO:teuthology.orchestra.run.smithi158.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-24T10:21:22.539 INFO:teuthology.lock.ops:Updating smithi158.front.sepia.ceph.com on lock server 2024-02-24T10:21:22.556 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:21:22.647 INFO:teuthology.dispatcher.supervisor:Running job 7572947 2024-02-24T10:21:22.647 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-24_10:11:26-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7572947 --name vshankar-2024-02-24_10:11:26-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi --description fs/fscrypt/{begin/{0-install 1-ceph 2-logrotate} bluestore-bitmap clusters/1-mds-1-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} overrides/{ignorelist_health ignorelist_health_more ignorelist_wrongly_marked_down osd pg-warn} tasks/{0-client 1-tests/fscrypt-common}} -- /home/teuthworker/archive/vshankar-2024-02-24_10:11:26-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7572947/orig.config.yaml 2024-02-24T10:21:22.650 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-24_10:11:26-fs-wip-vshankar-testing-20240216.060239-testing-default-smithi/7572947 2024-02-24T10:21:22.650 INFO:teuthology.dispatcher.supervisor:Job PID: 1476041 2024-02-24T10:21:22.650 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-24T10:23:22.724 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:25:22.884 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:27:23.052 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:29:23.216 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:31:23.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:33:23.536 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:35:23.704 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:37:23.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:39:24.017 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:41:24.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:43:24.352 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:45:24.517 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:47:24.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:49:24.832 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-24T10:49:24.907 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1