2024-04-11T23:35:23.019 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-11T23:35:23.019 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:35:23.234 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe505a33c10>, signals=[15]) 2024-04-11T23:35:23.234 INFO:teuthology.lock.ops:Start node 'smithi008.front.sepia.ceph.com' reimaging 2024-04-11T23:35:23.235 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: reset os type and version on server 2024-04-11T23:35:23.235 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2024-04-11T23:35:23.261 INFO:teuthology.lock.ops:Start node 'smithi150.front.sepia.ceph.com' reimaging 2024-04-11T23:35:23.261 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: reset os type and version on server 2024-04-11T23:35:23.261 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-11T23:35:23.286 INFO:teuthology.lock.ops:Start node 'smithi191.front.sepia.ceph.com' reimaging 2024-04-11T23:35:23.286 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: reset os type and version on server 2024-04-11T23:35:23.286 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-04-11T23:35:23.385 DEBUG:teuthology.provision.fog.smithi008:Requesting image smithi_centos_8.stream (ID 3) 2024-04-11T23:35:23.406 DEBUG:teuthology.provision.fog.smithi191:Requesting image smithi_centos_8.stream (ID 3) 2024-04-11T23:35:23.409 DEBUG:teuthology.provision.fog.smithi150:Requesting image smithi_centos_8.stream (ID 3) 2024-04-11T23:35:23.439 INFO:teuthology.provision.fog.smithi008:Scheduling deploy of centos 8.stream 2024-04-11T23:35:23.456 INFO:teuthology.provision.fog.smithi191:Scheduling deploy of centos 8.stream 2024-04-11T23:35:23.459 INFO:teuthology.provision.fog.smithi150:Scheduling deploy of centos 8.stream 2024-04-11T23:35:23.690 INFO:teuthology.orchestra.console.smithi008:Power off 2024-04-11T23:35:23.691 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-11T23:35:23.708 INFO:teuthology.orchestra.console.smithi150:Power off 2024-04-11T23:35:23.708 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-11T23:35:23.720 INFO:teuthology.orchestra.console.smithi191:Power off 2024-04-11T23:35:23.720 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-11T23:35:23.729 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2024-04-11T23:35:23.729 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:23.738 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2024-04-11T23:35:23.738 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:23.746 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-04-11T23:35:23.746 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:23.755 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-04-11T23:35:23.760 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-11T23:35:23.769 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-04-11T23:35:27.758 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:27.770 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:27.778 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:27.872 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-04-11T23:35:27.880 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-11T23:35:27.887 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-04-11T23:35:31.876 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:31.883 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:31.889 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:31.985 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-04-11T23:35:31.991 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-11T23:35:31.997 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-04-11T23:35:32.086 INFO:teuthology.orchestra.console.smithi008:Power off completed 2024-04-11T23:35:32.187 INFO:teuthology.orchestra.console.smithi008:Power on 2024-04-11T23:35:32.187 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-11T23:35:32.212 DEBUG:teuthology.orchestra.console.smithi008:power on output: Chassis Power Control: Up/On 2024-04-11T23:35:32.213 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:32.234 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-04-11T23:35:35.994 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:36.003 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:36.105 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-04-11T23:35:36.111 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-04-11T23:35:36.206 INFO:teuthology.orchestra.console.smithi150:Power off completed 2024-04-11T23:35:36.212 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-04-11T23:35:36.235 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:36.307 INFO:teuthology.orchestra.console.smithi150:Power on 2024-04-11T23:35:36.308 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-11T23:35:36.315 INFO:teuthology.orchestra.console.smithi191:Power on 2024-04-11T23:35:36.315 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2024-04-11T23:35:36.332 DEBUG:teuthology.orchestra.console.smithi150:power on output: Chassis Power Control: Up/On 2024-04-11T23:35:36.332 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:36.341 DEBUG:teuthology.orchestra.console.smithi191:power on output: Chassis Power Control: Up/On 2024-04-11T23:35:36.341 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:36.350 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-04-11T23:35:36.355 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-04-11T23:35:36.365 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-04-11T23:35:40.354 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:40.363 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:40.371 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:40.474 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2024-04-11T23:35:40.480 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-04-11T23:35:40.530 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-04-11T23:35:40.581 INFO:teuthology.orchestra.console.smithi191:Power on completed 2024-04-11T23:35:40.631 INFO:teuthology.orchestra.console.smithi008:Power on completed 2024-04-11T23:35:40.682 INFO:teuthology.provision.fog.smithi191:Waiting for deploy to finish 2024-04-11T23:35:40.732 INFO:teuthology.provision.fog.smithi008:Waiting for deploy to finish 2024-04-11T23:35:44.478 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-11T23:35:44.588 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2024-04-11T23:35:44.688 INFO:teuthology.orchestra.console.smithi150:Power on completed 2024-04-11T23:35:44.789 INFO:teuthology.provision.fog.smithi150:Waiting for deploy to finish 2024-04-11T23:38:11.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:14.594 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-04-11T23:38:20.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:23.678 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-04-11T23:38:26.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:29.662 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-11T23:38:29.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:30.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:32.734 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-04-11T23:38:33.726 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-04-11T23:38:35.665 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:38.718 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-11T23:38:38.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:39.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:41.794 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-04-11T23:38:42.782 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-04-11T23:38:44.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:47.774 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-11T23:38:47.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:48.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:50.846 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-04-11T23:38:51.842 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-04-11T23:38:53.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:56.830 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-11T23:38:56.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:57.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:38:59.905 WARNING:teuthology.provision.fog.smithi008:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-04-11T23:39:00.929 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-04-11T23:39:02.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:39:05.890 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-11T23:39:05.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:39:06.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:39:09.982 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-04-11T23:39:11.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:39:14.974 WARNING:teuthology.provision.fog.smithi191:[Errno None] Unable to connect to port 22 on 172.21.15.191 2024-04-11T23:39:15.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:39:19.038 WARNING:teuthology.provision.fog.smithi150:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-04-11T23:39:20.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:39:25.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:40:05.950 WARNING:teuthology.provision.fog.smithi008:timed out 2024-04-11T23:40:11.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:40:12.207 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-11T23:40:12.208 DEBUG:teuthology.orchestra.run.smithi008:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-11T23:40:12.433 INFO:teuthology.provision.fog.smithi008:Node is ready 2024-04-11T23:40:12.433 DEBUG:teuthology.orchestra.run.smithi008:> hostname 2024-04-11T23:40:12.453 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008.front.sepia.ceph.com 2024-04-11T23:40:12.453 DEBUG:teuthology.orchestra.run.smithi008:> grep smithi008.front.sepia.ceph.com /etc/hosts 2024-04-11T23:40:12.514 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8 smithi008.front.sepia.ceph.com smithi008 2024-04-11T23:40:12.514 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname smithi008 2024-04-11T23:40:12.723 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/smithi008.front.sepia.ceph.com/smithi008/g' /etc/hosts 2024-04-11T23:40:12.755 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/172.21.15.8/172.21.15.8/g' /etc/hosts 2024-04-11T23:40:12.827 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i -e 's/smithi008.front.sepia.ceph.com/smithi008/g' /etc/hostname 2024-04-11T23:40:12.898 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname smithi008 2024-04-11T23:40:12.968 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2024-04-11T23:40:13.028 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-11T23:40:13.029 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-11T23:40:13.030 INFO:teuthology.provision.fog.smithi008:Deploy complete! 2024-04-11T23:40:13.032 DEBUG:teuthology.parallel:result is None 2024-04-11T23:40:20.986 WARNING:teuthology.provision.fog.smithi191:timed out 2024-04-11T23:40:25.048 WARNING:teuthology.provision.fog.smithi150:timed out 2024-04-11T23:40:26.990 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:40:27.292 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-11T23:40:27.292 DEBUG:teuthology.orchestra.run.smithi191:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-11T23:40:27.560 INFO:teuthology.provision.fog.smithi191:Node is ready 2024-04-11T23:40:27.560 DEBUG:teuthology.orchestra.run.smithi191:> hostname 2024-04-11T23:40:27.579 INFO:teuthology.orchestra.run.smithi191.stdout:smithi191.front.sepia.ceph.com 2024-04-11T23:40:27.579 DEBUG:teuthology.orchestra.run.smithi191:> grep smithi191.front.sepia.ceph.com /etc/hosts 2024-04-11T23:40:27.638 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.15.191 smithi191.front.sepia.ceph.com smithi191 2024-04-11T23:40:27.639 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191 2024-04-11T23:40:27.747 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hosts 2024-04-11T23:40:27.776 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/172.21.15.191/172.21.15.191/g' /etc/hosts 2024-04-11T23:40:27.845 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i -e 's/smithi191.front.sepia.ceph.com/smithi191/g' /etc/hostname 2024-04-11T23:40:27.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname smithi191 2024-04-11T23:40:27.980 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-04-11T23:40:28.039 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-04-11T23:40:28.039 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8" 2024-04-11T23:40:28.039 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-04-11T23:40:28.039 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-04-11T23:40:28.039 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8" 2024-04-11T23:40:28.039 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2024-04-11T23:40:28.039 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-11T23:40:28.039 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-04-11T23:40:28.040 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-11T23:40:28.040 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-04-11T23:40:28.040 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-11T23:40:28.040 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-11T23:40:28.040 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-11T23:40:28.040 INFO:teuthology.provision.fog.smithi191:Deploy complete! 2024-04-11T23:40:28.041 DEBUG:teuthology.parallel:result is None 2024-04-11T23:40:31.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:40:31.339 INFO:teuthology.orchestra.run:Running command with timeout 600 2024-04-11T23:40:31.339 DEBUG:teuthology.orchestra.run.smithi150:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2024-04-11T23:40:31.605 INFO:teuthology.provision.fog.smithi150:Node is ready 2024-04-11T23:40:31.606 DEBUG:teuthology.orchestra.run.smithi150:> hostname 2024-04-11T23:40:31.625 INFO:teuthology.orchestra.run.smithi150.stdout:smithi150.front.sepia.ceph.com 2024-04-11T23:40:31.625 DEBUG:teuthology.orchestra.run.smithi150:> grep smithi150.front.sepia.ceph.com /etc/hosts 2024-04-11T23:40:31.685 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.15.150 smithi150.front.sepia.ceph.com smithi150 2024-04-11T23:40:31.685 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname smithi150 2024-04-11T23:40:31.786 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/smithi150.front.sepia.ceph.com/smithi150/g' /etc/hosts 2024-04-11T23:40:31.819 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/172.21.15.150/172.21.15.150/g' /etc/hosts 2024-04-11T23:40:31.888 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i -e 's/smithi150.front.sepia.ceph.com/smithi150/g' /etc/hostname 2024-04-11T23:40:31.958 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname smithi150 2024-04-11T23:40:32.026 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-11T23:40:32.085 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-11T23:40:32.085 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8" 2024-04-11T23:40:32.085 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-11T23:40:32.085 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-11T23:40:32.085 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8" 2024-04-11T23:40:32.085 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2024-04-11T23:40:32.086 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-11T23:40:32.086 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-11T23:40:32.086 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-11T23:40:32.086 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-11T23:40:32.086 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-11T23:40:32.086 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-11T23:40:32.086 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-11T23:40:32.086 INFO:teuthology.provision.fog.smithi150:Deploy complete! 2024-04-11T23:40:32.087 DEBUG:teuthology.parallel:result is None 2024-04-11T23:40:32.308 INFO:teuthology.lock.ops:Checking smithi008.front.sepia.ceph.com 2024-04-11T23:40:32.309 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-11T23:40:32.335 INFO:teuthology.lock.ops:Checking smithi150.front.sepia.ceph.com 2024-04-11T23:40:32.335 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-11T23:40:32.351 INFO:teuthology.lock.ops:Checking smithi191.front.sepia.ceph.com 2024-04-11T23:40:32.351 INFO:teuthology.lock.ops:New key found. Updating... 2024-04-11T23:40:32.369 INFO:teuthology.lock.ops:Updating [smithi008.front.sepia.ceph.com]: set os type and version on server 2024-04-11T23:40:32.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-04-11T23:40:32.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:40:32.445 DEBUG:teuthology.orchestra.run.smithi008:> true 2024-04-11T23:40:32.581 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-04-11T23:40:32.582 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2024-04-11T23:40:32.643 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2024-04-11T23:40:32.643 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2024-04-11T23:40:32.704 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-11T23:40:32.705 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-11T23:40:32.706 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2024-04-11T23:40:32.727 INFO:teuthology.lock.ops:Updating [smithi150.front.sepia.ceph.com]: set os type and version on server 2024-04-11T23:40:32.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-11T23:40:32.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:40:32.801 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-11T23:40:32.869 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-11T23:40:32.869 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-04-11T23:40:32.927 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-04-11T23:40:32.928 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-11T23:40:32.986 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-11T23:40:32.986 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8" 2024-04-11T23:40:32.986 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-11T23:40:32.986 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-11T23:40:32.986 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8" 2024-04-11T23:40:32.987 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2024-04-11T23:40:32.987 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-11T23:40:32.987 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-11T23:40:32.987 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-11T23:40:32.987 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-11T23:40:32.987 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-11T23:40:32.987 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-11T23:40:32.987 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-11T23:40:32.987 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-11T23:40:33.010 INFO:teuthology.lock.ops:Updating [smithi191.front.sepia.ceph.com]: set os type and version on server 2024-04-11T23:40:33.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-11T23:40:33.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-11T23:40:33.082 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-04-11T23:40:33.160 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-04-11T23:40:33.160 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-04-11T23:40:33.222 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-04-11T23:40:33.222 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-04-11T23:40:33.282 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-04-11T23:40:33.282 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8" 2024-04-11T23:40:33.282 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-04-11T23:40:33.282 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-04-11T23:40:33.282 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8" 2024-04-11T23:40:33.282 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2024-04-11T23:40:33.282 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-11T23:40:33.283 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-04-11T23:40:33.283 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-11T23:40:33.283 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-04-11T23:40:33.283 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-11T23:40:33.283 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-11T23:40:33.283 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-11T23:40:33.283 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-04-11T23:40:33.301 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:40:33.398 INFO:teuthology.dispatcher.supervisor:Running job 7652416 2024-04-11T23:40:33.398 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/virtualenv/bin/teuthology -v --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-04-11_17:03:56-upgrade-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652416 --name yuriw-2024-04-11_17:03:56-upgrade-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi --description upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_8.stream clusters/1-mds-2-client-micro conf/{client mds mgr mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down multimds/no multimds/yes pg-warn pg_health} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/quincy}} -- /home/teuthworker/archive/yuriw-2024-04-11_17:03:56-upgrade-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652416/orig.config.yaml 2024-04-11T23:40:33.401 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/yuriw-2024-04-11_17:03:56-upgrade-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652416 2024-04-11T23:40:33.402 INFO:teuthology.dispatcher.supervisor:Job PID: 4160512 2024-04-11T23:40:33.402 INFO:teuthology.dispatcher.supervisor:Running with watchdog 2024-04-11T23:42:33.503 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:44:33.677 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:46:33.843 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:48:34.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:50:34.171 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:52:34.343 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:54:34.515 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:56:34.631 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-11T23:58:34.806 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T00:00:34.967 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T00:02:35.160 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T00:04:35.334 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T00:06:35.500 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T00:06:35.572 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1