2024-02-14T12:51:27.940 INFO:root:teuthology version: 0.0.1.dev258+g9e44c16 2024-02-14T12:51:27.941 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T12:51:28.121 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f86aca1fc10>, signals=[15]) 2024-02-14T12:51:28.122 INFO:teuthology.lock.ops:Start node 'smithi044.front.sepia.ceph.com' reimaging 2024-02-14T12:51:28.123 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: reset os type and version on server 2024-02-14T12:51:28.123 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-02-14T12:51:28.147 INFO:teuthology.lock.ops:Start node 'smithi072.front.sepia.ceph.com' reimaging 2024-02-14T12:51:28.147 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: reset os type and version on server 2024-02-14T12:51:28.147 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-02-14T12:51:28.171 INFO:teuthology.lock.ops:Start node 'smithi089.front.sepia.ceph.com' reimaging 2024-02-14T12:51:28.172 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: reset os type and version on server 2024-02-14T12:51:28.172 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-02-14T12:51:28.293 DEBUG:teuthology.provision.fog.smithi072:Requesting image smithi_centos_8.stream (ID 3) 2024-02-14T12:51:28.314 DEBUG:teuthology.provision.fog.smithi089:Requesting image smithi_centos_8.stream (ID 3) 2024-02-14T12:51:28.339 INFO:teuthology.provision.fog.smithi072:Scheduling deploy of centos 8.stream 2024-02-14T12:51:28.347 DEBUG:teuthology.provision.fog.smithi044:Requesting image smithi_centos_8.stream (ID 3) 2024-02-14T12:51:28.358 INFO:teuthology.provision.fog.smithi089:Scheduling deploy of centos 8.stream 2024-02-14T12:51:28.393 INFO:teuthology.provision.fog.smithi044:Scheduling deploy of centos 8.stream 2024-02-14T12:51:28.537 INFO:teuthology.orchestra.console.smithi072:Power off 2024-02-14T12:51:28.537 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T12:51:28.559 DEBUG:teuthology.orchestra.console.smithi072:power off output: Chassis Power Control: Down/Off 2024-02-14T12:51:28.559 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:28.575 INFO:teuthology.orchestra.console.smithi089:Power off 2024-02-14T12:51:28.576 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T12:51:28.586 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-14T12:51:28.597 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2024-02-14T12:51:28.598 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:28.614 INFO:teuthology.orchestra.console.smithi044:Power off 2024-02-14T12:51:28.614 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-14T12:51:28.622 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-02-14T12:51:28.635 DEBUG:teuthology.orchestra.console.smithi044:power off output: Chassis Power Control: Down/Off 2024-02-14T12:51:28.635 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:28.659 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-02-14T12:51:32.588 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:32.623 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:32.660 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:32.699 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-14T12:51:32.732 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-02-14T12:51:32.770 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-02-14T12:51:36.704 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:36.732 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:36.771 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:36.813 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-14T12:51:36.841 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-02-14T12:51:36.881 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-02-14T12:51:36.914 INFO:teuthology.orchestra.console.smithi072:Power off completed 2024-02-14T12:51:37.015 INFO:teuthology.orchestra.console.smithi072:Power on 2024-02-14T12:51:37.015 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-14T12:51:37.037 DEBUG:teuthology.orchestra.console.smithi072:power on output: Chassis Power Control: Up/On 2024-02-14T12:51:37.037 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:37.059 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-14T12:51:40.844 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:40.881 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:40.954 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2024-02-14T12:51:40.990 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-02-14T12:51:41.055 INFO:teuthology.orchestra.console.smithi089:Power off completed 2024-02-14T12:51:41.060 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:41.090 INFO:teuthology.orchestra.console.smithi044:Power off completed 2024-02-14T12:51:41.156 INFO:teuthology.orchestra.console.smithi089:Power on 2024-02-14T12:51:41.156 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-14T12:51:41.169 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is off 2024-02-14T12:51:41.176 DEBUG:teuthology.orchestra.console.smithi089:power on output: Chassis Power Control: Up/On 2024-02-14T12:51:41.177 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:41.191 INFO:teuthology.orchestra.console.smithi044:Power on 2024-02-14T12:51:41.192 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-02-14T12:51:41.199 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2024-02-14T12:51:41.214 DEBUG:teuthology.orchestra.console.smithi044:power on output: Chassis Power Control: Up/On 2024-02-14T12:51:41.214 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:41.235 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-02-14T12:51:45.172 DEBUG:teuthology.orchestra.console.smithi072:pexpect command: ipmitool -H smithi072.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:45.201 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:45.236 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:45.311 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-02-14T12:51:45.345 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is off 2024-02-14T12:51:45.390 DEBUG:teuthology.orchestra.console.smithi072:check power output: Chassis Power is on 2024-02-14T12:51:45.413 INFO:teuthology.orchestra.console.smithi089:Power on completed 2024-02-14T12:51:45.492 INFO:teuthology.orchestra.console.smithi072:Power on completed 2024-02-14T12:51:45.514 INFO:teuthology.provision.fog.smithi089:Waiting for deploy to finish 2024-02-14T12:51:45.593 INFO:teuthology.provision.fog.smithi072:Waiting for deploy to finish 2024-02-14T12:51:49.349 DEBUG:teuthology.orchestra.console.smithi044:pexpect command: ipmitool -H smithi044.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-14T12:51:49.458 DEBUG:teuthology.orchestra.console.smithi044:check power output: Chassis Power is on 2024-02-14T12:51:49.559 INFO:teuthology.orchestra.console.smithi044:Power on completed 2024-02-14T12:51:49.660 INFO:teuthology.provision.fog.smithi044:Waiting for deploy to finish 2024-02-14T12:54:20.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:23.455 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-02-14T12:54:29.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:31.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:32.540 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-02-14T12:54:34.492 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-02-14T12:54:38.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:40.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:41.596 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-02-14T12:54:43.548 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-02-14T12:54:47.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:49.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:50.652 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-02-14T12:54:52.672 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-02-14T12:54:56.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:58.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:54:59.708 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-02-14T12:55:01.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:01.760 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-02-14T12:55:04.572 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T12:55:05.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:07.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:08.768 WARNING:teuthology.provision.fog.smithi044:[Errno None] Unable to connect to port 22 on 172.21.15.44 2024-02-14T12:55:10.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:10.844 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-02-14T12:55:13.632 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T12:55:14.770 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:16.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:19.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:19.904 WARNING:teuthology.provision.fog.smithi089:[Errno None] Unable to connect to port 22 on 172.21.15.89 2024-02-14T12:55:22.716 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T12:55:25.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:28.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:31.776 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T12:55:37.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:40.864 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T12:55:46.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:55:49.948 WARNING:teuthology.provision.fog.smithi072:[Errno None] Unable to connect to port 22 on 172.21.15.72 2024-02-14T12:55:55.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:56:14.788 WARNING:teuthology.provision.fog.smithi044:timed out 2024-02-14T12:56:20.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:56:21.066 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-14T12:56:21.066 DEBUG:teuthology.orchestra.run.smithi044:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-14T12:56:21.301 INFO:teuthology.provision.fog.smithi044:Node is ready 2024-02-14T12:56:21.301 DEBUG:teuthology.orchestra.run.smithi044:> hostname 2024-02-14T12:56:21.320 INFO:teuthology.orchestra.run.smithi044.stdout:smithi044.front.sepia.ceph.com 2024-02-14T12:56:21.321 DEBUG:teuthology.orchestra.run.smithi044:> grep smithi044.front.sepia.ceph.com /etc/hosts 2024-02-14T12:56:21.381 INFO:teuthology.orchestra.run.smithi044.stdout:172.21.15.44 smithi044.front.sepia.ceph.com smithi044 2024-02-14T12:56:21.381 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2024-02-14T12:56:21.474 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hosts 2024-02-14T12:56:21.504 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/172.21.15.44/172.21.15.44/g' /etc/hosts 2024-02-14T12:56:21.573 DEBUG:teuthology.orchestra.run.smithi044:> sudo sed -i -e 's/smithi044.front.sepia.ceph.com/smithi044/g' /etc/hostname 2024-02-14T12:56:21.642 DEBUG:teuthology.orchestra.run.smithi044:> sudo hostname smithi044 2024-02-14T12:56:21.710 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-14T12:56:21.768 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T12:56:21.769 INFO:teuthology.provision.fog.smithi044:Deploy complete! 2024-02-14T12:56:21.788 DEBUG:teuthology.parallel:result is None 2024-02-14T12:56:25.915 WARNING:teuthology.provision.fog.smithi089:timed out 2024-02-14T12:56:31.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:56:32.130 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-14T12:56:32.131 DEBUG:teuthology.orchestra.run.smithi089:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-14T12:56:32.364 INFO:teuthology.provision.fog.smithi089:Node is ready 2024-02-14T12:56:32.364 DEBUG:teuthology.orchestra.run.smithi089:> hostname 2024-02-14T12:56:32.382 INFO:teuthology.orchestra.run.smithi089.stdout:smithi089.front.sepia.ceph.com 2024-02-14T12:56:32.382 DEBUG:teuthology.orchestra.run.smithi089:> grep smithi089.front.sepia.ceph.com /etc/hosts 2024-02-14T12:56:32.441 INFO:teuthology.orchestra.run.smithi089.stdout:172.21.15.89 smithi089.front.sepia.ceph.com smithi089 2024-02-14T12:56:32.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089 2024-02-14T12:56:32.545 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hosts 2024-02-14T12:56:32.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/172.21.15.89/172.21.15.89/g' /etc/hosts 2024-02-14T12:56:32.641 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i -e 's/smithi089.front.sepia.ceph.com/smithi089/g' /etc/hostname 2024-02-14T12:56:32.709 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname smithi089 2024-02-14T12:56:32.776 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-02-14T12:56:32.834 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-14T12:56:32.835 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T12:56:32.836 INFO:teuthology.provision.fog.smithi089:Deploy complete! 2024-02-14T12:56:32.836 DEBUG:teuthology.parallel:result is None 2024-02-14T12:56:55.980 WARNING:teuthology.provision.fog.smithi072:timed out 2024-02-14T12:57:01.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:57:02.265 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-02-14T12:57:02.265 DEBUG:teuthology.orchestra.run.smithi072:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-02-14T12:57:02.542 INFO:teuthology.provision.fog.smithi072:Node is ready 2024-02-14T12:57:02.542 DEBUG:teuthology.orchestra.run.smithi072:> hostname 2024-02-14T12:57:02.561 INFO:teuthology.orchestra.run.smithi072.stdout:smithi072.front.sepia.ceph.com 2024-02-14T12:57:02.561 DEBUG:teuthology.orchestra.run.smithi072:> grep smithi072.front.sepia.ceph.com /etc/hosts 2024-02-14T12:57:02.620 INFO:teuthology.orchestra.run.smithi072.stdout:172.21.15.72 smithi072.front.sepia.ceph.com smithi072 2024-02-14T12:57:02.620 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname smithi072 2024-02-14T12:57:02.742 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i -e 's/smithi072.front.sepia.ceph.com/smithi072/g' /etc/hosts 2024-02-14T12:57:02.771 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i -e 's/172.21.15.72/172.21.15.72/g' /etc/hosts 2024-02-14T12:57:02.839 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i -e 's/smithi072.front.sepia.ceph.com/smithi072/g' /etc/hostname 2024-02-14T12:57:02.908 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname smithi072 2024-02-14T12:57:02.975 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-02-14T12:57:03.032 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T12:57:03.033 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-14T12:57:03.034 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T12:57:03.034 INFO:teuthology.provision.fog.smithi072:Deploy complete! 2024-02-14T12:57:03.034 DEBUG:teuthology.parallel:result is None 2024-02-14T12:57:03.219 INFO:teuthology.lock.ops:Checking smithi044.front.sepia.ceph.com 2024-02-14T12:57:03.220 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-14T12:57:03.244 INFO:teuthology.lock.ops:Checking smithi072.front.sepia.ceph.com 2024-02-14T12:57:03.244 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-14T12:57:03.260 INFO:teuthology.lock.ops:Checking smithi089.front.sepia.ceph.com 2024-02-14T12:57:03.260 INFO:teuthology.lock.ops:New key found. Updating... 2024-02-14T12:57:03.278 INFO:teuthology.lock.ops:Updating [smithi044.front.sepia.ceph.com]: set os type and version on server 2024-02-14T12:57:03.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-02-14T12:57:03.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:57:03.353 DEBUG:teuthology.orchestra.run.smithi044:> true 2024-02-14T12:57:03.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi044.front.sepia.ceph.com' 2024-02-14T12:57:03.496 DEBUG:teuthology.orchestra.run.smithi044:> uname -m 2024-02-14T12:57:03.557 INFO:teuthology.orchestra.run.smithi044.stdout:x86_64 2024-02-14T12:57:03.557 DEBUG:teuthology.orchestra.run.smithi044:> cat /etc/os-release 2024-02-14T12:57:03.616 INFO:teuthology.orchestra.run.smithi044.stdout:NAME="CentOS Stream" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION="8" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:ID="centos" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:ID_LIKE="rhel fedora" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:VERSION_ID="8" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:PLATFORM_ID="platform:el8" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:ANSI_COLOR="0;31" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:HOME_URL="https://centos.org/" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-14T12:57:03.617 INFO:teuthology.orchestra.run.smithi044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T12:57:03.618 INFO:teuthology.lock.ops:Updating smithi044.front.sepia.ceph.com on lock server 2024-02-14T12:57:03.636 INFO:teuthology.lock.ops:Updating [smithi072.front.sepia.ceph.com]: set os type and version on server 2024-02-14T12:57:03.636 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-02-14T12:57:03.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:57:03.704 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-02-14T12:57:03.770 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-02-14T12:57:03.770 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2024-02-14T12:57:03.828 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2024-02-14T12:57:03.828 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="8" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="8" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el8" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T12:57:03.886 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-14T12:57:03.887 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T12:57:03.887 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-02-14T12:57:03.911 INFO:teuthology.lock.ops:Updating [smithi089.front.sepia.ceph.com]: set os type and version on server 2024-02-14T12:57:03.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-02-14T12:57:03.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-14T12:57:03.982 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-02-14T12:57:04.114 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-02-14T12:57:04.114 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-02-14T12:57:04.173 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-02-14T12:57:04.173 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="8" 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="8" 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el8" 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 8" 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-02-14T12:57:04.232 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-02-14T12:57:04.233 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-02-14T12:57:04.233 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-14T12:57:04.233 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-14T12:57:04.233 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-14T12:57:04.233 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-02-14T12:57:04.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T12:57:04.335 INFO:teuthology.dispatcher.supervisor:Running job 7559603 2024-02-14T12:57:04.336 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_9e44c16382dfc4c612f7437cbb344c76223137cc/virtualenv/bin/teuthology -v --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559603 --name vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi --description fs/upgrade/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no pg-warn} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} -- /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559603/orig.config.yaml 2024-02-14T12:57:04.338 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/vshankar-2024-02-14_05:24:17-fs-wip-vshankar-testing-20240206.051942-testing-default-smithi/7559603 2024-02-14T12:57:04.339 INFO:teuthology.dispatcher.supervisor:Job PID: 1240871 2024-02-14T12:57:04.339 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-02-14T12:59:04.439 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T13:01:04.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-14T13:01:04.764 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1