2024-03-18T23:51:18.341 INFO:root:teuthology version: 0.0.1.dev262+ge691533
2024-03-18T23:51:18.342 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T23:51:18.523 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f7aa4dd1f40>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f7aa5d6fc10>, signals=[15])
2024-03-18T23:51:18.524 INFO:teuthology.lock.ops:Start node 'smithi087.front.sepia.ceph.com' reimaging
2024-03-18T23:51:18.524 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: reset os type and version on server
2024-03-18T23:51:18.524 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server
2024-03-18T23:51:18.545 INFO:teuthology.lock.ops:Start node 'smithi154.front.sepia.ceph.com' reimaging
2024-03-18T23:51:18.545 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: reset os type and version on server
2024-03-18T23:51:18.545 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2024-03-18T23:51:18.569 INFO:teuthology.lock.ops:Start node 'smithi164.front.sepia.ceph.com' reimaging
2024-03-18T23:51:18.570 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: reset os type and version on server
2024-03-18T23:51:18.570 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2024-03-18T23:51:18.663 DEBUG:teuthology.provision.fog.smithi087:Requesting image smithi_centos_9.stream (ID 10)
2024-03-18T23:51:18.693 DEBUG:teuthology.provision.fog.smithi154:Requesting image smithi_centos_9.stream (ID 10)
2024-03-18T23:51:18.710 DEBUG:teuthology.provision.fog.smithi164:Requesting image smithi_centos_9.stream (ID 10)
2024-03-18T23:51:18.713 INFO:teuthology.provision.fog.smithi087:Scheduling deploy of centos 9.stream
2024-03-18T23:51:18.730 INFO:teuthology.provision.fog.smithi154:Scheduling deploy of centos 9.stream
2024-03-18T23:51:18.755 INFO:teuthology.provision.fog.smithi164:Scheduling deploy of centos 9.stream
2024-03-18T23:51:18.908 INFO:teuthology.orchestra.console.smithi087:Power off
2024-03-18T23:51:18.909 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-18T23:51:18.923 INFO:teuthology.orchestra.console.smithi154:Power off
2024-03-18T23:51:18.923 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-18T23:51:18.935 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off
2024-03-18T23:51:18.935 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:18.948 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off
2024-03-18T23:51:18.948 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:18.956 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off
2024-03-18T23:51:18.968 INFO:teuthology.orchestra.console.smithi164:Power off
2024-03-18T23:51:18.969 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2024-03-18T23:51:18.975 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on
2024-03-18T23:51:18.990 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off
2024-03-18T23:51:18.990 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:19.011 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-03-18T23:51:19.057 INFO:teuthology.orchestra.console.smithi087:Power off completed
2024-03-18T23:51:19.157 INFO:teuthology.orchestra.console.smithi087:Power on
2024-03-18T23:51:19.157 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-18T23:51:19.179 DEBUG:teuthology.orchestra.console.smithi087:power on output: Chassis Power Control: Up/On
2024-03-18T23:51:19.180 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:19.200 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off
2024-03-18T23:51:22.979 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:23.013 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:23.089 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on
2024-03-18T23:51:23.122 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-03-18T23:51:23.201 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:23.309 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off
2024-03-18T23:51:27.094 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:27.123 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:27.203 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off
2024-03-18T23:51:27.232 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-03-18T23:51:27.304 INFO:teuthology.orchestra.console.smithi154:Power off completed
2024-03-18T23:51:27.310 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:27.333 INFO:teuthology.orchestra.console.smithi164:Power off completed
2024-03-18T23:51:27.405 INFO:teuthology.orchestra.console.smithi154:Power on
2024-03-18T23:51:27.405 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-18T23:51:27.430 DEBUG:teuthology.orchestra.console.smithi154:power on output: Chassis Power Control: Up/On
2024-03-18T23:51:27.430 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:27.438 INFO:teuthology.orchestra.console.smithi164:Power on
2024-03-18T23:51:27.439 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2024-03-18T23:51:27.445 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on
2024-03-18T23:51:27.453 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off
2024-03-18T23:51:27.458 DEBUG:teuthology.orchestra.console.smithi164:power on output: Chassis Power Control: Up/On
2024-03-18T23:51:27.459 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:27.480 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-03-18T23:51:27.546 INFO:teuthology.orchestra.console.smithi087:Power on completed
2024-03-18T23:51:27.647 INFO:teuthology.provision.fog.smithi087:Waiting for deploy to finish
2024-03-18T23:51:31.458 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:31.480 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:31.568 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off
2024-03-18T23:51:31.589 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off
2024-03-18T23:51:35.573 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:35.589 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2024-03-18T23:51:35.682 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on
2024-03-18T23:51:35.705 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on
2024-03-18T23:51:35.783 INFO:teuthology.orchestra.console.smithi154:Power on completed
2024-03-18T23:51:35.806 INFO:teuthology.orchestra.console.smithi164:Power on completed
2024-03-18T23:51:35.884 INFO:teuthology.provision.fog.smithi154:Waiting for deploy to finish
2024-03-18T23:51:35.908 INFO:teuthology.provision.fog.smithi164:Waiting for deploy to finish
2024-03-18T23:53:58.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:53:58.387 ERROR:teuthology.orchestra.connection:Error authenticating with smithi087.front.sepia.ceph.com: AuthenticationException('Authentication failed.')
2024-03-18T23:54:06.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:09.630 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-03-18T23:54:15.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:18.690 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-03-18T23:54:21.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:24.670 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2024-03-18T23:54:24.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:27.742 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-03-18T23:54:30.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:33.726 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2024-03-18T23:54:33.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:36.802 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-03-18T23:54:36.958 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87
2024-03-18T23:54:39.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:42.782 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2024-03-18T23:54:42.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:42.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:45.854 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-03-18T23:54:46.018 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87
2024-03-18T23:54:48.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:51.842 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2024-03-18T23:54:51.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:52.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:54:54.914 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-03-18T23:54:55.070 WARNING:teuthology.provision.fog.smithi087:[Errno None] Unable to connect to port 22 on 172.21.15.87
2024-03-18T23:54:57.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:55:00.898 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2024-03-18T23:55:00.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:55:01.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:55:03.966 WARNING:teuthology.provision.fog.smithi164:[Errno None] Unable to connect to port 22 on 172.21.15.164
2024-03-18T23:55:06.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:55:09.950 WARNING:teuthology.provision.fog.smithi154:[Errno None] Unable to connect to port 22 on 172.21.15.154
2024-03-18T23:55:09.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:55:15.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:56:01.074 WARNING:teuthology.provision.fog.smithi087:timed out
2024-03-18T23:56:07.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:56:07.341 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-18T23:56:07.341 DEBUG:teuthology.orchestra.run.smithi087:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-18T23:56:07.653 INFO:teuthology.provision.fog.smithi087:Node is ready
2024-03-18T23:56:07.653 DEBUG:teuthology.orchestra.run.smithi087:> hostname
2024-03-18T23:56:07.667 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087.front.sepia.ceph.com
2024-03-18T23:56:07.668 DEBUG:teuthology.orchestra.run.smithi087:> grep smithi087.front.sepia.ceph.com /etc/hosts
2024-03-18T23:56:07.723 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87 smithi087.front.sepia.ceph.com smithi087
2024-03-18T23:56:07.723 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087
2024-03-18T23:56:07.839 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087.front.sepia.ceph.com/smithi087/g' /etc/hosts
2024-03-18T23:56:07.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/172.21.15.87/172.21.15.87/g' /etc/hosts
2024-03-18T23:56:07.934 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i -e 's/smithi087.front.sepia.ceph.com/smithi087/g' /etc/hostname
2024-03-18T23:56:08.002 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname smithi087
2024-03-18T23:56:08.068 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release
2024-03-18T23:56:08.122 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream"
2024-03-18T23:56:08.122 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9"
2024-03-18T23:56:08.122 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T23:56:08.123 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T23:56:08.124 INFO:teuthology.provision.fog.smithi087:Deploy complete!
2024-03-18T23:56:08.125 DEBUG:teuthology.parallel:result is None
2024-03-18T23:56:09.970 WARNING:teuthology.provision.fog.smithi164:timed out
2024-03-18T23:56:15.954 WARNING:teuthology.provision.fog.smithi154:timed out
2024-03-18T23:56:15.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:56:16.214 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-18T23:56:16.214 DEBUG:teuthology.orchestra.run.smithi164:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-18T23:56:16.550 INFO:teuthology.provision.fog.smithi164:Node is ready
2024-03-18T23:56:16.550 DEBUG:teuthology.orchestra.run.smithi164:> hostname
2024-03-18T23:56:16.564 INFO:teuthology.orchestra.run.smithi164.stdout:smithi164.front.sepia.ceph.com
2024-03-18T23:56:16.564 DEBUG:teuthology.orchestra.run.smithi164:> grep smithi164.front.sepia.ceph.com /etc/hosts
2024-03-18T23:56:16.618 INFO:teuthology.orchestra.run.smithi164.stdout:172.21.15.164 smithi164.front.sepia.ceph.com smithi164
2024-03-18T23:56:16.619 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2024-03-18T23:56:16.724 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hosts
2024-03-18T23:56:16.751 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/172.21.15.164/172.21.15.164/g' /etc/hosts
2024-03-18T23:56:16.818 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i -e 's/smithi164.front.sepia.ceph.com/smithi164/g' /etc/hostname
2024-03-18T23:56:16.885 DEBUG:teuthology.orchestra.run.smithi164:> sudo hostname smithi164
2024-03-18T23:56:16.951 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2024-03-18T23:56:17.005 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T23:56:17.006 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T23:56:17.007 INFO:teuthology.provision.fog.smithi164:Deploy complete!
2024-03-18T23:56:17.007 DEBUG:teuthology.parallel:result is None
2024-03-18T23:56:21.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:56:22.396 INFO:teuthology.orchestra.run:Running command with timeout 600
2024-03-18T23:56:22.396 DEBUG:teuthology.orchestra.run.smithi154:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done
2024-03-18T23:56:22.706 INFO:teuthology.provision.fog.smithi154:Node is ready
2024-03-18T23:56:22.706 DEBUG:teuthology.orchestra.run.smithi154:> hostname
2024-03-18T23:56:22.723 INFO:teuthology.orchestra.run.smithi154.stdout:smithi154.front.sepia.ceph.com
2024-03-18T23:56:22.723 DEBUG:teuthology.orchestra.run.smithi154:> grep smithi154.front.sepia.ceph.com /etc/hosts
2024-03-18T23:56:22.779 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154 smithi154.front.sepia.ceph.com smithi154
2024-03-18T23:56:22.779 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154
2024-03-18T23:56:22.882 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hosts
2024-03-18T23:56:22.909 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/172.21.15.154/172.21.15.154/g' /etc/hosts
2024-03-18T23:56:22.977 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i -e 's/smithi154.front.sepia.ceph.com/smithi154/g' /etc/hostname
2024-03-18T23:56:23.044 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname smithi154
2024-03-18T23:56:23.110 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release
2024-03-18T23:56:23.164 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream"
2024-03-18T23:56:23.164 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9"
2024-03-18T23:56:23.164 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T23:56:23.165 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T23:56:23.165 INFO:teuthology.provision.fog.smithi154:Deploy complete!
2024-03-18T23:56:23.166 DEBUG:teuthology.parallel:result is None
2024-03-18T23:56:23.372 INFO:teuthology.lock.ops:Checking smithi087.front.sepia.ceph.com
2024-03-18T23:56:23.373 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-18T23:56:23.393 INFO:teuthology.lock.ops:Checking smithi154.front.sepia.ceph.com
2024-03-18T23:56:23.393 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-18T23:56:23.419 INFO:teuthology.lock.ops:Checking smithi164.front.sepia.ceph.com
2024-03-18T23:56:23.419 INFO:teuthology.lock.ops:New key found. Updating...
2024-03-18T23:56:23.436 INFO:teuthology.lock.ops:Updating [smithi087.front.sepia.ceph.com]: set os type and version on server
2024-03-18T23:56:23.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com'
2024-03-18T23:56:23.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:56:23.527 DEBUG:teuthology.orchestra.run.smithi087:> true
2024-03-18T23:56:23.700 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com'
2024-03-18T23:56:23.700 DEBUG:teuthology.orchestra.run.smithi087:> uname -m
2024-03-18T23:56:23.756 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64
2024-03-18T23:56:23.756 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release
2024-03-18T23:56:23.810 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream"
2024-03-18T23:56:23.810 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9"
2024-03-18T23:56:23.810 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos"
2024-03-18T23:56:23.810 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora"
2024-03-18T23:56:23.810 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9"
2024-03-18T23:56:23.810 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9"
2024-03-18T23:56:23.810 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T23:56:23.810 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31"
2024-03-18T23:56:23.811 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon"
2024-03-18T23:56:23.811 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T23:56:23.811 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/"
2024-03-18T23:56:23.811 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T23:56:23.811 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T23:56:23.811 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T23:56:23.811 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server
2024-03-18T23:56:23.835 INFO:teuthology.lock.ops:Updating [smithi154.front.sepia.ceph.com]: set os type and version on server
2024-03-18T23:56:23.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com'
2024-03-18T23:56:23.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:56:23.922 DEBUG:teuthology.orchestra.run.smithi154:> true
2024-03-18T23:56:23.988 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com'
2024-03-18T23:56:23.989 DEBUG:teuthology.orchestra.run.smithi154:> uname -m
2024-03-18T23:56:24.043 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64
2024-03-18T23:56:24.043 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release
2024-03-18T23:56:24.098 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T23:56:24.099 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T23:56:24.100 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server
2024-03-18T23:56:24.127 INFO:teuthology.lock.ops:Updating [smithi164.front.sepia.ceph.com]: set os type and version on server
2024-03-18T23:56:24.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com'
2024-03-18T23:56:24.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2024-03-18T23:56:24.229 DEBUG:teuthology.orchestra.run.smithi164:> true
2024-03-18T23:56:24.299 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com'
2024-03-18T23:56:24.299 DEBUG:teuthology.orchestra.run.smithi164:> uname -m
2024-03-18T23:56:24.355 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64
2024-03-18T23:56:24.355 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release
2024-03-18T23:56:24.412 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream"
2024-03-18T23:56:24.412 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9"
2024-03-18T23:56:24.412 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos"
2024-03-18T23:56:24.412 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora"
2024-03-18T23:56:24.412 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2024-03-18T23:56:24.413 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2024-03-18T23:56:24.413 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server
2024-03-18T23:56:24.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-18T23:56:24.548 INFO:teuthology.dispatcher.supervisor:Running job 7609239
2024-03-18T23:56:24.549 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609239 --name pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/direct_io}} -- /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609239/orig.config.yaml
2024-03-18T23:56:24.552 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2024-03-18_21:05:49-fs-wip-batrick-testing-20240318.181317-distro-default-smithi/7609239
2024-03-18T23:56:24.552 INFO:teuthology.dispatcher.supervisor:Job PID: 2914915
2024-03-18T23:56:24.553 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-18T23:58:24.554 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:00:24.649 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:02:24.712 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:04:24.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:06:24.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:08:24.913 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:10:24.987 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:12:25.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:14:25.131 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:16:25.206 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:18:25.283 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:20:25.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:22:25.445 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:24:25.503 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:26:25.595 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:28:25.661 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:30:25.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:32:25.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:34:25.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:36:26.011 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:38:26.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:40:26.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:42:26.242 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:44:26.321 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:46:26.395 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:48:26.477 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:50:26.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:52:26.599 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:54:26.673 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:56:26.748 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T00:58:26.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:00:26.897 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:02:26.969 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:04:27.047 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:06:27.113 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:08:27.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:10:27.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:12:27.318 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:14:27.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:16:27.457 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:18:27.529 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:20:27.612 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-19T01:20:27.689 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1