2024-04-14T10:12:57.469 INFO:root:teuthology version: 0.0.1.dev265+gd7d6e60 2024-04-14T10:12:57.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:12:57.637 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa9e95b3c10>, signals=[15]) 2024-04-14T10:12:57.637 INFO:teuthology.lock.ops:Start node 'smithi023.front.sepia.ceph.com' reimaging 2024-04-14T10:12:57.638 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T10:12:57.638 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2024-04-14T10:12:57.654 INFO:teuthology.lock.ops:Start node 'smithi055.front.sepia.ceph.com' reimaging 2024-04-14T10:12:57.654 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T10:12:57.654 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-04-14T10:12:57.670 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging 2024-04-14T10:12:57.670 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server 2024-04-14T10:12:57.670 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-04-14T10:12:57.773 DEBUG:teuthology.provision.fog.smithi055:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T10:12:57.777 DEBUG:teuthology.provision.fog.smithi023:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T10:12:57.809 DEBUG:teuthology.provision.fog.smithi191:Requesting image smithi_centos_9.stream (ID 10) 2024-04-14T10:12:57.822 INFO:teuthology.provision.fog.smithi023:Scheduling deploy of centos 9.stream 2024-04-14T10:12:57.826 INFO:teuthology.provision.fog.smithi055:Scheduling deploy of centos 9.stream 2024-04-14T10:12:57.855 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of centos 9.stream 2024-04-14T10:12:58.147 INFO:teuthology.orchestra.console.smithi055:Power off 2024-04-14T10:12:58.148 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T10:12:58.162 INFO:teuthology.orchestra.console.smithi191:Power off 2024-04-14T10:12:58.162 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T10:12:58.174 INFO:teuthology.orchestra.console.smithi023:Power off 2024-04-14T10:12:58.175 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-14T10:12:58.184 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2024-04-14T10:12:58.184 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:12:58.193 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-04-14T10:12:58.193 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:12:58.200 DEBUG:teuthology.orchestra.console.smithi023:power off output: Chassis Power Control: Down/Off 2024-04-14T10:12:58.201 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:12:58.213 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-04-14T10:12:58.217 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-04-14T10:12:58.223 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-04-14T10:13:02.215 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:02.222 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:02.228 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:02.325 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-04-14T10:13:02.329 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-04-14T10:13:02.336 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-04-14T10:13:06.329 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:06.335 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:06.344 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:06.438 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-04-14T10:13:06.446 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-04-14T10:13:06.454 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-04-14T10:13:06.539 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-04-14T10:13:06.639 INFO:teuthology.orchestra.console.smithi191:Power on 2024-04-14T10:13:06.640 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T10:13:06.663 DEBUG:teuthology.orchestra.console.smithi191:power on output: Chassis Power Control: Up/On 2024-04-14T10:13:06.663 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:06.687 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-04-14T10:13:10.448 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:10.456 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:10.558 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-04-14T10:13:10.565 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2024-04-14T10:13:10.659 INFO:teuthology.orchestra.console.smithi055:Power off completed 2024-04-14T10:13:10.666 INFO:teuthology.orchestra.console.smithi023:Power off completed 2024-04-14T10:13:10.688 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:10.760 INFO:teuthology.orchestra.console.smithi055:Power on 2024-04-14T10:13:10.761 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T10:13:10.768 INFO:teuthology.orchestra.console.smithi023:Power on 2024-04-14T10:13:10.768 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-14T10:13:10.782 DEBUG:teuthology.orchestra.console.smithi055:power on output: Chassis Power Control: Up/On 2024-04-14T10:13:10.782 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:10.789 DEBUG:teuthology.orchestra.console.smithi023:power on output: Chassis Power Control: Up/On 2024-04-14T10:13:10.790 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:10.798 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-04-14T10:13:10.802 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-04-14T10:13:10.810 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is off 2024-04-14T10:13:14.802 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:14.809 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:14.815 DEBUG:teuthology.orchestra.console.smithi023:pexpect command: ipmitool -H smithi023.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:14.911 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-04-14T10:13:14.917 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-04-14T10:13:14.923 DEBUG:teuthology.orchestra.console.smithi023:check power output: Chassis Power is on 2024-04-14T10:13:15.012 INFO:teuthology.orchestra.console.smithi191:Power on completed 2024-04-14T10:13:15.024 INFO:teuthology.orchestra.console.smithi023:Power on completed 2024-04-14T10:13:15.113 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish 2024-04-14T10:13:15.124 INFO:teuthology.provision.fog.smithi023:Waiting for deploy to finish 2024-04-14T10:13:18.921 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-14T10:13:19.031 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-04-14T10:13:19.131 INFO:teuthology.orchestra.console.smithi055:Power on completed 2024-04-14T10:13:19.232 INFO:teuthology.provision.fog.smithi055:Waiting for deploy to finish 2024-04-14T10:15:34.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:15:37.950 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-04-14T10:15:43.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:15:45.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:15:47.038 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-04-14T10:15:48.958 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-14T10:15:53.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:15:54.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:15:56.094 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-04-14T10:15:58.014 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-14T10:16:02.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:04.017 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:05.154 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-04-14T10:16:07.070 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-14T10:16:11.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:13.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:14.238 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-04-14T10:16:16.130 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-14T10:16:20.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:22.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:23.298 WARNING:teuthology.provision.fog.smithi055:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-04-14T10:16:25.214 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-14T10:16:29.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:31.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:31.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:34.174 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:16:34.274 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-14T10:16:40.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:40.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:43.266 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:16:49.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:16:52.350 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:16:58.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:17:00.996 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T10:17:00.996 DEBUG:teuthology.orchestra.run.smithi055:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T10:17:01.406 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:17:07.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:17:10.466 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:17:16.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:17:17.180 INFO:teuthology.provision.fog.smithi055:Node is ready 2024-04-14T10:17:17.181 DEBUG:teuthology.orchestra.run.smithi055:> hostname 2024-04-14T10:17:17.200 INFO:teuthology.orchestra.run.smithi055.stdout:smithi046 2024-04-14T10:17:17.201 DEBUG:teuthology.orchestra.run.smithi055:> grep smithi046 /etc/hosts 2024-04-14T10:17:17.258 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.15.46 smithi046.front.sepia.ceph.com smithi046 2024-04-14T10:17:17.258 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname smithi055 2024-04-14T10:17:17.421 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi046/smithi055/g' /etc/hosts 2024-04-14T10:17:17.455 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/172.21.15.46/172.21.15.55/g' /etc/hosts 2024-04-14T10:17:17.529 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i -e 's/smithi046/smithi055/g' /etc/hostname 2024-04-14T10:17:17.602 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname smithi055 2024-04-14T10:17:17.674 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-04-14T10:17:17.730 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-04-14T10:17:17.730 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T10:17:17.731 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T10:17:17.732 INFO:teuthology.provision.fog.smithi055:Deploy complete! 2024-04-14T10:17:17.734 DEBUG:teuthology.parallel:result is None 2024-04-14T10:17:19.518 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:17:25.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:17:28.578 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:17:34.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:17:37.666 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:17:40.278 WARNING:teuthology.provision.fog.smithi191:timed out 2024-04-14T10:17:43.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:17:46.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:17:46.521 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T10:17:46.521 DEBUG:teuthology.orchestra.run.smithi191:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T10:17:46.750 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:17:46.855 INFO:teuthology.provision.fog.smithi191:Node is ready 2024-04-14T10:17:46.855 DEBUG:teuthology.orchestra.run.smithi191:> hostname 2024-04-14T10:17:46.870 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com 2024-04-14T10:17:46.870 DEBUG:teuthology.orchestra.run.smithi191:> grep smithi191.front.sepia.ceph.com /etc/hosts 2024-04-14T10:17:46.926 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191 2024-04-14T10:17:46.927 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191 2024-04-14T10:17:47.056 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hosts 2024-04-14T10:17:47.085 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/172.21.15.191/172.21.15.191/g' /etc/hosts 2024-04-14T10:17:47.155 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hostname 2024-04-14T10:17:47.222 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191 2024-04-14T10:17:47.289 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-04-14T10:17:47.343 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-04-14T10:17:47.343 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-04-14T10:17:47.343 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-04-14T10:17:47.343 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-04-14T10:17:47.343 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-04-14T10:17:47.343 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-04-14T10:17:47.343 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T10:17:47.344 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-04-14T10:17:47.344 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-04-14T10:17:47.344 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T10:17:47.344 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-04-14T10:17:47.344 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T10:17:47.344 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T10:17:47.344 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T10:17:47.344 INFO:teuthology.provision.fog.smithi191:Deploy complete! 2024-04-14T10:17:47.345 DEBUG:teuthology.parallel:result is None 2024-04-14T10:17:52.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:17:55.806 WARNING:teuthology.provision.fog.smithi023:[Errno None] Unable to connect to port 22 on 172.21.15.23 2024-04-14T10:18:01.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:19:01.875 WARNING:teuthology.provision.fog.smithi023:timed out 2024-04-14T10:19:07.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:19:08.119 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-14T10:19:08.119 DEBUG:teuthology.orchestra.run.smithi023:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-14T10:19:08.463 INFO:teuthology.provision.fog.smithi023:Node is ready 2024-04-14T10:19:08.463 DEBUG:teuthology.orchestra.run.smithi023:> hostname 2024-04-14T10:19:08.479 INFO:teuthology.orchestra.run.smithi023.stdout:smithi023.front.sepia.ceph.com 2024-04-14T10:19:08.479 DEBUG:teuthology.orchestra.run.smithi023:> grep smithi023.front.sepia.ceph.com /etc/hosts 2024-04-14T10:19:08.535 INFO:teuthology.orchestra.run.smithi023.stdout:172.21.15.23 smithi023.front.sepia.ceph.com smithi023 2024-04-14T10:19:08.535 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023 2024-04-14T10:19:08.655 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hosts 2024-04-14T10:19:08.688 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/172.21.15.23/172.21.15.23/g' /etc/hosts 2024-04-14T10:19:08.759 DEBUG:teuthology.orchestra.run.smithi023:> sudo sed -i -e 's/smithi023.front.sepia.ceph.com/smithi023/g' /etc/hostname 2024-04-14T10:19:08.829 DEBUG:teuthology.orchestra.run.smithi023:> sudo hostname smithi023 2024-04-14T10:19:08.900 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2024-04-14T10:19:08.955 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2024-04-14T10:19:08.955 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9" 2024-04-14T10:19:08.955 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2024-04-14T10:19:08.955 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2024-04-14T10:19:08.955 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9" 2024-04-14T10:19:08.955 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9" 2024-04-14T10:19:08.955 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T10:19:08.956 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2024-04-14T10:19:08.956 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon" 2024-04-14T10:19:08.956 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T10:19:08.956 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2024-04-14T10:19:08.956 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T10:19:08.956 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T10:19:08.956 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T10:19:08.956 INFO:teuthology.provision.fog.smithi023:Deploy complete! 2024-04-14T10:19:08.957 DEBUG:teuthology.parallel:result is None 2024-04-14T10:19:09.182 INFO:teuthology.lock.ops:Checking smithi023.front.sepia.ceph.com 2024-04-14T10:19:09.182 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T10:19:09.205 INFO:teuthology.lock.ops:Checking smithi055.front.sepia.ceph.com 2024-04-14T10:19:09.205 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T10:19:09.222 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com 2024-04-14T10:19:09.223 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-14T10:19:09.239 INFO:teuthology.lock.ops:Updating [smithi023.front.sepia.ceph.com]: set os type and version on server 2024-04-14T10:19:09.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-04-14T10:19:09.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:19:09.343 DEBUG:teuthology.orchestra.run.smithi023:> true 2024-04-14T10:19:09.409 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi023.front.sepia.ceph.com' 2024-04-14T10:19:09.409 DEBUG:teuthology.orchestra.run.smithi023:> uname -m 2024-04-14T10:19:09.464 INFO:teuthology.orchestra.run.smithi023.stdout:x86_64 2024-04-14T10:19:09.464 DEBUG:teuthology.orchestra.run.smithi023:> cat /etc/os-release 2024-04-14T10:19:09.518 INFO:teuthology.orchestra.run.smithi023.stdout:NAME="CentOS Stream" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION="9" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:ID="centos" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:ID_LIKE="rhel fedora" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:VERSION_ID="9" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:PLATFORM_ID="platform:el9" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:ANSI_COLOR="0;31" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:LOGO="fedora-logo-icon" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:HOME_URL="https://centos.org/" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T10:19:09.519 INFO:teuthology.orchestra.run.smithi023.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T10:19:09.520 INFO:teuthology.lock.ops:Updating smithi023.front.sepia.ceph.com on lock server 2024-04-14T10:19:09.539 INFO:teuthology.lock.ops:Updating [smithi055.front.sepia.ceph.com]: set os type and version on server 2024-04-14T10:19:09.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-04-14T10:19:09.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:19:09.648 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-04-14T10:19:09.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-04-14T10:19:09.833 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2024-04-14T10:19:09.889 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2024-04-14T10:19:09.889 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-04-14T10:19:09.945 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-04-14T10:19:09.945 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-04-14T10:19:09.945 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-04-14T10:19:09.945 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-04-14T10:19:09.945 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T10:19:09.946 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T10:19:09.947 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-04-14T10:19:09.971 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server 2024-04-14T10:19:09.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-14T10:19:09.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-14T10:19:10.063 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-04-14T10:19:10.233 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-14T10:19:10.233 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-04-14T10:19:10.288 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-04-14T10:19:10.288 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-04-14T10:19:10.345 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-14T10:19:10.346 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-04-14T10:19:10.346 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-14T10:19:10.346 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-14T10:19:10.346 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-14T10:19:10.346 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-04-14T10:19:10.372 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:19:10.470 INFO:teuthology.dispatcher.supervisor:Running job 7638556 2024-04-14T10:19:10.471 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_d7d6e60867cadf19fbfd7655a551ae7d58060757/virtualenv/bin/teuthology -v --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638556 --name teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/fs/misc}} -- /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638556/orig.config.yaml 2024-04-14T10:19:10.474 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/teuthology-2024-04-03_20:24:15-fs-main-distro-default-smithi/7638556 2024-04-14T10:19:10.474 INFO:teuthology.dispatcher.supervisor:Job PID: 2813217 2024-04-14T10:19:10.474 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-14T10:21:10.575 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:23:10.739 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:25:10.895 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:27:11.071 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:29:11.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:31:11.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:33:11.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:35:11.715 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:37:11.879 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-14T10:37:11.948 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1