2024-04-19T13:21:03.702 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-19T13:21:03.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-19T13:21:03.847 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f93b2965c10>, signals=[15]) 2024-04-19T13:21:03.850 INFO:teuthology.lock.ops:Start node 'smithi118.front.sepia.ceph.com' reimaging 2024-04-19T13:21:03.850 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: reset os type and version on server 2024-04-19T13:21:03.851 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-04-19T13:21:03.909 INFO:teuthology.lock.ops:Start node 'smithi122.front.sepia.ceph.com' reimaging 2024-04-19T13:21:03.909 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: reset os type and version on server 2024-04-19T13:21:03.909 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-04-19T13:21:03.928 INFO:teuthology.lock.ops:Start node 'smithi159.front.sepia.ceph.com' reimaging 2024-04-19T13:21:03.929 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: reset os type and version on server 2024-04-19T13:21:03.929 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-04-19T13:21:04.016 DEBUG:teuthology.provision.fog.smithi122:Requesting image smithi_centos_9.stream (ID 10) 2024-04-19T13:21:04.021 DEBUG:teuthology.provision.fog.smithi118:Requesting image smithi_centos_9.stream (ID 10) 2024-04-19T13:21:04.062 DEBUG:teuthology.provision.fog.smithi159:Requesting image smithi_centos_9.stream (ID 10) 2024-04-19T13:21:04.073 INFO:teuthology.provision.fog.smithi118:Scheduling deploy of centos 9.stream 2024-04-19T13:21:04.075 INFO:teuthology.provision.fog.smithi122:Scheduling deploy of centos 9.stream 2024-04-19T13:21:04.105 INFO:teuthology.provision.fog.smithi159:Scheduling deploy of centos 9.stream 2024-04-19T13:21:04.594 INFO:teuthology.orchestra.console.smithi118:Power off 2024-04-19T13:21:04.594 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-19T13:21:04.615 DEBUG:teuthology.orchestra.console.smithi118:power off output: Chassis Power Control: Down/Off 2024-04-19T13:21:04.616 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:04.628 INFO:teuthology.orchestra.console.smithi122:Power off 2024-04-19T13:21:04.628 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-19T13:21:04.639 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-04-19T13:21:04.652 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-04-19T13:21:04.652 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:04.669 INFO:teuthology.orchestra.console.smithi159:Power off 2024-04-19T13:21:04.669 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-19T13:21:04.678 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-19T13:21:04.691 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-04-19T13:21:04.692 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:04.712 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-04-19T13:21:04.740 INFO:teuthology.orchestra.console.smithi118:Power off completed 2024-04-19T13:21:04.813 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-04-19T13:21:04.841 INFO:teuthology.orchestra.console.smithi118:Power on 2024-04-19T13:21:04.841 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-19T13:21:04.862 DEBUG:teuthology.orchestra.console.smithi118:power on output: Chassis Power Control: Up/On 2024-04-19T13:21:04.863 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:04.883 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-04-19T13:21:04.914 INFO:teuthology.orchestra.console.smithi159:Power on 2024-04-19T13:21:04.914 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-19T13:21:04.934 DEBUG:teuthology.orchestra.console.smithi159:power on output: Chassis Power Control: Up/On 2024-04-19T13:21:04.935 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:04.959 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-04-19T13:21:08.682 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:08.789 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-19T13:21:08.884 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:08.960 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:08.994 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is off 2024-04-19T13:21:09.068 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-04-19T13:21:12.794 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:12.902 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-04-19T13:21:12.995 DEBUG:teuthology.orchestra.console.smithi118:pexpect command: ipmitool -H smithi118.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:13.003 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-04-19T13:21:13.069 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:13.103 DEBUG:teuthology.orchestra.console.smithi118:check power output: Chassis Power is on 2024-04-19T13:21:13.104 INFO:teuthology.orchestra.console.smithi122:Power on 2024-04-19T13:21:13.105 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-19T13:21:13.127 DEBUG:teuthology.orchestra.console.smithi122:power on output: Chassis Power Control: Up/On 2024-04-19T13:21:13.127 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:13.147 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-04-19T13:21:13.177 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-04-19T13:21:13.204 INFO:teuthology.orchestra.console.smithi118:Power on completed 2024-04-19T13:21:13.278 INFO:teuthology.orchestra.console.smithi159:Power on completed 2024-04-19T13:21:13.305 INFO:teuthology.provision.fog.smithi118:Waiting for deploy to finish 2024-04-19T13:21:13.379 INFO:teuthology.provision.fog.smithi159:Waiting for deploy to finish 2024-04-19T13:21:17.150 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:17.258 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-04-19T13:21:21.262 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-19T13:21:21.479 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-19T13:21:21.581 INFO:teuthology.orchestra.console.smithi122:Power on completed 2024-04-19T13:21:21.682 INFO:teuthology.provision.fog.smithi122:Waiting for deploy to finish 2024-04-19T13:24:36.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:24:39.774 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-19T13:24:45.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:24:48.830 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-19T13:24:54.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:24:56.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:24:57.886 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-19T13:24:59.742 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-19T13:25:03.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:05.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:06.942 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-19T13:25:08.798 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-19T13:25:12.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:14.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:15.998 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-19T13:25:17.854 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-19T13:25:22.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:23.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:25.054 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-19T13:25:26.910 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-19T13:25:31.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:31.190 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:32.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:34.110 WARNING:teuthology.provision.fog.smithi122:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-19T13:25:34.270 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-19T13:25:35.966 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-19T13:25:40.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:40.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:41.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:43.326 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-19T13:25:45.022 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-19T13:25:49.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:51.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:25:52.382 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-19T13:25:54.078 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-19T13:25:58.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:00.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:01.438 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-19T13:26:03.134 WARNING:teuthology.provision.fog.smithi118:[Errno None] Unable to connect to port 22 on 172.21.15.118 2024-04-19T13:26:07.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:09.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:10.526 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-19T13:26:16.530 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:19.582 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-19T13:26:25.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:28.670 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-19T13:26:34.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:37.726 WARNING:teuthology.provision.fog.smithi159:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-04-19T13:26:40.119 WARNING:teuthology.provision.fog.smithi122:timed out 2024-04-19T13:26:43.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:46.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:26:46.473 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-19T13:26:46.473 DEBUG:teuthology.orchestra.run.smithi122:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-19T13:26:46.792 INFO:teuthology.provision.fog.smithi122:Node is ready 2024-04-19T13:26:46.792 DEBUG:teuthology.orchestra.run.smithi122:> hostname 2024-04-19T13:26:46.806 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122.front.sepia.ceph.com 2024-04-19T13:26:46.806 DEBUG:teuthology.orchestra.run.smithi122:> grep smithi122.front.sepia.ceph.com /etc/hosts 2024-04-19T13:26:46.864 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122 smithi122.front.sepia.ceph.com smithi122 2024-04-19T13:26:46.865 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122 2024-04-19T13:26:46.989 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi122.front.sepia.ceph.com/smithi122/g' /etc/hosts 2024-04-19T13:26:47.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/172.21.15.122/172.21.15.122/g' /etc/hosts 2024-04-19T13:26:47.086 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i -e 's/smithi122.front.sepia.ceph.com/smithi122/g' /etc/hostname 2024-04-19T13:26:47.155 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname smithi122 2024-04-19T13:26:47.225 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-04-19T13:26:47.283 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-04-19T13:26:47.283 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-04-19T13:26:47.283 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-04-19T13:26:47.283 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-19T13:26:47.284 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-19T13:26:47.285 INFO:teuthology.provision.fog.smithi122:Deploy complete! 2024-04-19T13:26:47.286 DEBUG:teuthology.parallel:result is None 2024-04-19T13:27:09.156 WARNING:teuthology.provision.fog.smithi118:timed out 2024-04-19T13:27:15.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:27:15.691 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-19T13:27:15.691 DEBUG:teuthology.orchestra.run.smithi118:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-19T13:27:16.029 INFO:teuthology.provision.fog.smithi118:Node is ready 2024-04-19T13:27:16.029 DEBUG:teuthology.orchestra.run.smithi118:> hostname 2024-04-19T13:27:16.045 INFO:teuthology.orchestra.run.smithi118.stdout:smithi118.front.sepia.ceph.com 2024-04-19T13:27:16.045 DEBUG:teuthology.orchestra.run.smithi118:> grep smithi118.front.sepia.ceph.com /etc/hosts 2024-04-19T13:27:16.099 INFO:teuthology.orchestra.run.smithi118.stdout:172.21.15.118 smithi118.front.sepia.ceph.com smithi118 2024-04-19T13:27:16.100 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2024-04-19T13:27:16.210 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hosts 2024-04-19T13:27:16.240 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/172.21.15.118/172.21.15.118/g' /etc/hosts 2024-04-19T13:27:16.309 DEBUG:teuthology.orchestra.run.smithi118:> sudo sed -i -e 's/smithi118.front.sepia.ceph.com/smithi118/g' /etc/hostname 2024-04-19T13:27:16.376 DEBUG:teuthology.orchestra.run.smithi118:> sudo hostname smithi118 2024-04-19T13:27:16.442 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-04-19T13:27:16.495 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-04-19T13:27:16.496 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-19T13:27:16.496 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-04-19T13:27:16.496 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-19T13:27:16.496 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-19T13:27:16.496 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-19T13:27:16.496 INFO:teuthology.provision.fog.smithi118:Deploy complete! 2024-04-19T13:27:16.496 DEBUG:teuthology.parallel:result is None 2024-04-19T13:27:43.747 WARNING:teuthology.provision.fog.smithi159:timed out 2024-04-19T13:27:49.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:27:50.020 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-19T13:27:50.020 DEBUG:teuthology.orchestra.run.smithi159:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-19T13:27:50.337 INFO:teuthology.provision.fog.smithi159:Node is ready 2024-04-19T13:27:50.337 DEBUG:teuthology.orchestra.run.smithi159:> hostname 2024-04-19T13:27:50.352 INFO:teuthology.orchestra.run.smithi159.stdout:smithi159.front.sepia.ceph.com 2024-04-19T13:27:50.352 DEBUG:teuthology.orchestra.run.smithi159:> grep smithi159.front.sepia.ceph.com /etc/hosts 2024-04-19T13:27:50.404 INFO:teuthology.orchestra.run.smithi159.stdout:172.21.15.159 smithi159.front.sepia.ceph.com smithi159 2024-04-19T13:27:50.405 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2024-04-19T13:27:50.509 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hosts 2024-04-19T13:27:50.537 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/172.21.15.159/172.21.15.159/g' /etc/hosts 2024-04-19T13:27:50.602 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i -e 's/smithi159.front.sepia.ceph.com/smithi159/g' /etc/hostname 2024-04-19T13:27:50.669 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname smithi159 2024-04-19T13:27:50.732 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-04-19T13:27:50.785 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-04-19T13:27:50.785 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-19T13:27:50.786 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-19T13:27:50.786 INFO:teuthology.provision.fog.smithi159:Deploy complete! 2024-04-19T13:27:50.787 DEBUG:teuthology.parallel:result is None 2024-04-19T13:27:51.001 INFO:teuthology.lock.ops:Checking smithi118.front.sepia.ceph.com 2024-04-19T13:27:51.001 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-19T13:27:51.038 INFO:teuthology.lock.ops:Checking smithi122.front.sepia.ceph.com 2024-04-19T13:27:51.038 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-19T13:27:51.063 INFO:teuthology.lock.ops:Checking smithi159.front.sepia.ceph.com 2024-04-19T13:27:51.063 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-19T13:27:51.080 INFO:teuthology.lock.ops:Updating [smithi118.front.sepia.ceph.com]: set os type and version on server 2024-04-19T13:27:51.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-04-19T13:27:51.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:27:51.165 DEBUG:teuthology.orchestra.run.smithi118:> true 2024-04-19T13:27:51.332 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi118.front.sepia.ceph.com' 2024-04-19T13:27:51.332 DEBUG:teuthology.orchestra.run.smithi118:> uname -m 2024-04-19T13:27:51.386 INFO:teuthology.orchestra.run.smithi118.stdout:x86_64 2024-04-19T13:27:51.387 DEBUG:teuthology.orchestra.run.smithi118:> cat /etc/os-release 2024-04-19T13:27:51.441 INFO:teuthology.orchestra.run.smithi118.stdout:NAME="CentOS Stream" 2024-04-19T13:27:51.441 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION="9" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:ID="centos" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:ID_LIKE="rhel fedora" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:VERSION_ID="9" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:PLATFORM_ID="platform:el9" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:ANSI_COLOR="0;31" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:LOGO="fedora-logo-icon" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:HOME_URL="https://centos.org/" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-19T13:27:51.442 INFO:teuthology.orchestra.run.smithi118.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-19T13:27:51.443 INFO:teuthology.lock.ops:Updating smithi118.front.sepia.ceph.com on lock server 2024-04-19T13:27:51.568 INFO:teuthology.lock.ops:Updating [smithi122.front.sepia.ceph.com]: set os type and version on server 2024-04-19T13:27:51.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-19T13:27:51.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:27:51.656 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-19T13:27:51.824 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-19T13:27:51.825 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-04-19T13:27:51.879 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-04-19T13:27:51.879 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-04-19T13:27:51.933 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-19T13:27:51.934 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-19T13:27:51.934 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-19T13:27:51.934 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-04-19T13:27:51.963 INFO:teuthology.lock.ops:Updating [smithi159.front.sepia.ceph.com]: set os type and version on server 2024-04-19T13:27:51.963 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-19T13:27:51.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-19T13:27:52.051 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-04-19T13:27:52.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-04-19T13:27:52.123 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-04-19T13:27:52.177 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-04-19T13:27:52.177 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-04-19T13:27:52.232 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-04-19T13:27:52.232 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-04-19T13:27:52.232 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-19T13:27:52.233 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-19T13:27:52.233 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-04-19T13:27:52.263 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-19T13:27:52.344 INFO:teuthology.dispatcher.supervisor:Running job 7663463 2024-04-19T13:27:52.344 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/bin/teuthology -v --owner scheduled_adking@teuthology --archive /home/teuthworker/archive/adking-2024-04-19_13:19:23-orch:cephadm-wip-adk-testing-2024-04-17-1713-distro-default-smithi/7663463 --name adking-2024-04-19_13:19:23-orch:cephadm-wip-adk-testing-2024-04-17-1713-distro-default-smithi --description orch:cephadm/workunits/{0-distro/centos_9.stream agent/off mon_election/classic task/test_rgw_multisite} -- /home/teuthworker/archive/adking-2024-04-19_13:19:23-orch:cephadm-wip-adk-testing-2024-04-17-1713-distro-default-smithi/7663463/orig.config.yaml 2024-04-19T13:27:52.347 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/adking-2024-04-19_13:19:23-orch:cephadm-wip-adk-testing-2024-04-17-1713-distro-default-smithi/7663463 2024-04-19T13:27:52.347 INFO:teuthology.dispatcher.supervisor:Job PID: 653949 2024-04-19T13:27:52.347 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-19T13:29:52.444 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-19T13:31:52.610 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-19T13:33:52.768 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-19T13:35:52.938 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-19T13:35:53.002 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1