2024-04-02T14:26:45.200 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T14:26:45.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:26:45.400 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f17db7a6c10>, signals=[15]) 2024-04-02T14:26:45.400 INFO:teuthology.lock.ops:Start node 'smithi082.front.sepia.ceph.com' reimaging 2024-04-02T14:26:45.400 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: reset os type and version on server 2024-04-02T14:26:45.400 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-04-02T14:26:45.428 INFO:teuthology.lock.ops:Start node 'smithi123.front.sepia.ceph.com' reimaging 2024-04-02T14:26:45.428 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: reset os type and version on server 2024-04-02T14:26:45.428 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-04-02T14:26:45.452 INFO:teuthology.lock.ops:Start node 'smithi196.front.sepia.ceph.com' reimaging 2024-04-02T14:26:45.452 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: reset os type and version on server 2024-04-02T14:26:45.452 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-02T14:26:45.551 DEBUG:teuthology.provision.fog.smithi123:Requesting image smithi_centos_8.stream (ID 3) 2024-04-02T14:26:45.556 DEBUG:teuthology.provision.fog.smithi082:Requesting image smithi_centos_8.stream (ID 3) 2024-04-02T14:26:45.596 DEBUG:teuthology.provision.fog.smithi196:Requesting image smithi_centos_8.stream (ID 3) 2024-04-02T14:26:45.600 INFO:teuthology.provision.fog.smithi123:Scheduling deploy of centos 8.stream 2024-04-02T14:26:45.603 INFO:teuthology.provision.fog.smithi082:Scheduling deploy of centos 8.stream 2024-04-02T14:26:45.654 INFO:teuthology.provision.fog.smithi196:Scheduling deploy of centos 8.stream 2024-04-02T14:26:45.850 INFO:teuthology.orchestra.console.smithi082:Power off 2024-04-02T14:26:45.851 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:26:45.863 INFO:teuthology.orchestra.console.smithi123:Power off 2024-04-02T14:26:45.864 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:26:45.873 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-04-02T14:26:45.874 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:45.888 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2024-04-02T14:26:45.888 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:45.897 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-02T14:26:45.909 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-04-02T14:26:45.923 INFO:teuthology.orchestra.console.smithi196:Power off 2024-04-02T14:26:45.924 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:26:45.950 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-04-02T14:26:45.950 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:45.972 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-02T14:26:49.902 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:49.910 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:49.973 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:50.012 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-02T14:26:50.020 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-04-02T14:26:50.082 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-02T14:26:54.014 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:54.023 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:54.082 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:54.125 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-02T14:26:54.132 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2024-04-02T14:26:54.192 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-02T14:26:54.234 INFO:teuthology.orchestra.console.smithi123:Power off completed 2024-04-02T14:26:54.334 INFO:teuthology.orchestra.console.smithi123:Power on 2024-04-02T14:26:54.335 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-02T14:26:54.359 DEBUG:teuthology.orchestra.console.smithi123:power on output: Chassis Power Control: Up/On 2024-04-02T14:26:54.359 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:54.381 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2024-04-02T14:26:58.130 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:58.193 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:58.240 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-04-02T14:26:58.303 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-02T14:26:58.342 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-04-02T14:26:58.381 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:58.404 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-04-02T14:26:58.443 INFO:teuthology.orchestra.console.smithi082:Power on 2024-04-02T14:26:58.444 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-02T14:26:58.467 DEBUG:teuthology.orchestra.console.smithi082:power on output: Chassis Power Control: Up/On 2024-04-02T14:26:58.467 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:58.489 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2024-04-02T14:26:58.490 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-04-02T14:26:58.506 INFO:teuthology.orchestra.console.smithi196:Power on 2024-04-02T14:26:58.506 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-02T14:26:58.530 DEBUG:teuthology.orchestra.console.smithi196:power on output: Chassis Power Control: Up/On 2024-04-02T14:26:58.530 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:58.551 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-02T14:27:02.494 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:27:02.502 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:27:02.553 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:27:02.604 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2024-04-02T14:27:02.611 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-04-02T14:27:02.662 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-04-02T14:27:02.705 INFO:teuthology.orchestra.console.smithi123:Power on completed 2024-04-02T14:27:02.805 INFO:teuthology.provision.fog.smithi123:Waiting for deploy to finish 2024-04-02T14:27:06.614 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:27:06.663 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:27:06.725 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-04-02T14:27:06.773 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-04-02T14:27:06.826 INFO:teuthology.orchestra.console.smithi082:Power on completed 2024-04-02T14:27:06.874 INFO:teuthology.orchestra.console.smithi196:Power on completed 2024-04-02T14:27:06.927 INFO:teuthology.provision.fog.smithi082:Waiting for deploy to finish 2024-04-02T14:27:06.975 INFO:teuthology.provision.fog.smithi196:Waiting for deploy to finish 2024-04-02T14:29:37.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:29:37.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:29:40.798 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-02T14:29:40.799 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-02T14:29:46.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:29:46.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:29:48.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:29:49.854 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-02T14:29:49.855 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-02T14:29:51.774 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-02T14:29:55.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:29:55.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:29:57.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:29:58.910 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-02T14:29:58.911 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-02T14:30:00.830 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-02T14:30:04.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:04.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:06.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:07.966 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-02T14:30:07.967 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-02T14:30:09.886 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-02T14:30:13.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:13.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:15.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:17.022 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-02T14:30:17.023 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-02T14:30:18.946 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-02T14:30:23.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:23.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:24.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:26.082 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-02T14:30:26.083 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-02T14:30:27.998 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-02T14:30:32.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:32.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:34.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:35.166 WARNING:teuthology.provision.fog.smithi082:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-02T14:30:35.167 WARNING:teuthology.provision.fog.smithi196:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-04-02T14:30:37.059 WARNING:teuthology.provision.fog.smithi123:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-04-02T14:30:41.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:41.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:30:43.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:31:12.919 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-02T14:31:12.919 DEBUG:teuthology.orchestra.run.smithi082:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-02T14:31:38.635 INFO:teuthology.provision.fog.smithi082:Node is ready 2024-04-02T14:31:38.636 DEBUG:teuthology.orchestra.run.smithi082:> hostname 2024-04-02T14:31:38.660 INFO:teuthology.orchestra.run.smithi082.stdout:smithi082.front.sepia.ceph.com 2024-04-02T14:31:38.660 DEBUG:teuthology.orchestra.run.smithi082:> grep smithi082.front.sepia.ceph.com /etc/hosts 2024-04-02T14:31:38.722 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82 smithi082.front.sepia.ceph.com smithi082 2024-04-02T14:31:38.722 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname smithi082 2024-04-02T14:31:38.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/smithi082.front.sepia.ceph.com/smithi082/g' /etc/hosts 2024-04-02T14:31:38.874 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/172.21.15.82/172.21.15.82/g' /etc/hosts 2024-04-02T14:31:38.944 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i -e 's/smithi082.front.sepia.ceph.com/smithi082/g' /etc/hostname 2024-04-02T14:31:39.013 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname smithi082 2024-04-02T14:31:39.082 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-04-02T14:31:39.141 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-04-02T14:31:39.141 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-02T14:31:39.142 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:31:39.143 INFO:teuthology.provision.fog.smithi082:Deploy complete! 2024-04-02T14:31:39.145 DEBUG:teuthology.parallel:result is None 2024-04-02T14:31:41.177 WARNING:teuthology.provision.fog.smithi196:timed out 2024-04-02T14:31:43.063 WARNING:teuthology.provision.fog.smithi123:timed out 2024-04-02T14:31:47.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:31:47.435 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-02T14:31:47.435 DEBUG:teuthology.orchestra.run.smithi196:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-02T14:31:47.705 INFO:teuthology.provision.fog.smithi196:Node is ready 2024-04-02T14:31:47.706 DEBUG:teuthology.orchestra.run.smithi196:> hostname 2024-04-02T14:31:47.726 INFO:teuthology.orchestra.run.smithi196.stdout:smithi196.front.sepia.ceph.com 2024-04-02T14:31:47.726 DEBUG:teuthology.orchestra.run.smithi196:> grep smithi196.front.sepia.ceph.com /etc/hosts 2024-04-02T14:31:47.786 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.15.196 smithi196.front.sepia.ceph.com smithi196 2024-04-02T14:31:47.786 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2024-04-02T14:31:47.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hosts 2024-04-02T14:31:47.932 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/172.21.15.196/172.21.15.196/g' /etc/hosts 2024-04-02T14:31:48.001 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i -e 's/smithi196.front.sepia.ceph.com/smithi196/g' /etc/hostname 2024-04-02T14:31:48.069 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname smithi196 2024-04-02T14:31:48.137 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8" 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8" 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-02T14:31:48.196 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-02T14:31:48.197 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:31:48.197 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:31:48.197 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-02T14:31:48.197 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:31:48.197 INFO:teuthology.provision.fog.smithi196:Deploy complete! 2024-04-02T14:31:48.198 DEBUG:teuthology.parallel:result is None 2024-04-02T14:31:49.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:31:49.308 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-02T14:31:49.308 DEBUG:teuthology.orchestra.run.smithi123:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-02T14:31:49.609 INFO:teuthology.provision.fog.smithi123:Node is ready 2024-04-02T14:31:49.609 DEBUG:teuthology.orchestra.run.smithi123:> hostname 2024-04-02T14:31:49.631 INFO:teuthology.orchestra.run.smithi123.stdout:smithi123.front.sepia.ceph.com 2024-04-02T14:31:49.631 DEBUG:teuthology.orchestra.run.smithi123:> grep smithi123.front.sepia.ceph.com /etc/hosts 2024-04-02T14:31:49.693 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.15.123 smithi123.front.sepia.ceph.com smithi123 2024-04-02T14:31:49.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname smithi123 2024-04-02T14:31:49.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/smithi123.front.sepia.ceph.com/smithi123/g' /etc/hosts 2024-04-02T14:31:49.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/172.21.15.123/172.21.15.123/g' /etc/hosts 2024-04-02T14:31:49.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i -e 's/smithi123.front.sepia.ceph.com/smithi123/g' /etc/hostname 2024-04-02T14:31:49.997 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname smithi123 2024-04-02T14:31:50.080 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-04-02T14:31:50.147 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-02T14:31:50.148 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:31:50.149 INFO:teuthology.provision.fog.smithi123:Deploy complete! 2024-04-02T14:31:50.149 DEBUG:teuthology.parallel:result is None 2024-04-02T14:31:50.684 INFO:teuthology.lock.ops:Checking smithi082.front.sepia.ceph.com 2024-04-02T14:31:50.685 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-02T14:31:50.717 INFO:teuthology.lock.ops:Checking smithi123.front.sepia.ceph.com 2024-04-02T14:31:50.718 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-02T14:31:50.745 INFO:teuthology.lock.ops:Checking smithi196.front.sepia.ceph.com 2024-04-02T14:31:50.745 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-02T14:31:50.768 INFO:teuthology.lock.ops:Updating [smithi082.front.sepia.ceph.com]: set os type and version on server 2024-04-02T14:31:50.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-02T14:31:50.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:31:50.848 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-02T14:31:50.984 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-02T14:31:50.984 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-04-02T14:31:51.044 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-04-02T14:31:51.045 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-04-02T14:31:51.105 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-04-02T14:31:51.105 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8" 2024-04-02T14:31:51.105 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-04-02T14:31:51.105 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:31:51.105 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8" 2024-04-02T14:31:51.105 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2024-04-02T14:31:51.105 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-02T14:31:51.105 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-04-02T14:31:51.106 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-02T14:31:51.106 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:31:51.106 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:31:51.106 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-02T14:31:51.106 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:31:51.106 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-04-02T14:31:51.134 INFO:teuthology.lock.ops:Updating [smithi123.front.sepia.ceph.com]: set os type and version on server 2024-04-02T14:31:51.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-02T14:31:51.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:31:51.212 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-02T14:31:51.241 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-02T14:31:51.241 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-04-02T14:31:51.302 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-04-02T14:31:51.303 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-04-02T14:31:51.363 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-04-02T14:31:51.363 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8" 2024-04-02T14:31:51.363 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-04-02T14:31:51.363 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:31:51.363 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8" 2024-04-02T14:31:51.363 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2024-04-02T14:31:51.363 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-02T14:31:51.364 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-04-02T14:31:51.364 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-02T14:31:51.364 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:31:51.364 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:31:51.364 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-02T14:31:51.364 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:31:51.364 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-04-02T14:31:51.400 INFO:teuthology.lock.ops:Updating [smithi196.front.sepia.ceph.com]: set os type and version on server 2024-04-02T14:31:51.401 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-02T14:31:51.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T14:31:51.477 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-02T14:31:51.552 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-02T14:31:51.552 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-02T14:31:51.614 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-02T14:31:51.614 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="8" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="8" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el8" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-02T14:31:51.675 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-04-02T14:31:51.676 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T14:31:51.676 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-02T14:31:51.676 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T14:31:51.676 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-02T14:31:51.701 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:31:51.813 INFO:teuthology.dispatcher.supervisor:Running job 7635660 2024-04-02T14:31:51.813 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635660 --name pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} -- /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635660/orig.config.yaml 2024-04-02T14:31:51.816 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635660 2024-04-02T14:31:51.816 INFO:teuthology.dispatcher.supervisor:Job PID: 747265 2024-04-02T14:31:51.816 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-02T14:33:51.915 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:35:52.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:37:52.259 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:39:52.430 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:41:52.551 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:43:52.715 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:45:52.891 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:47:53.070 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:49:53.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:51:53.427 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:53:53.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:55:53.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:57:53.863 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:57:53.939 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1