2026-02-12T08:33:54.910 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062
2026-02-12T08:33:54.911 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:33:54.982 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:33:55.110 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x781f0caa6390>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x781f0c281c60>, signals=[15])
2026-02-12T08:33:55.110 INFO:teuthology.lock.ops:Start node 'trial098.front.sepia.ceph.com' reimaging
2026-02-12T08:33:55.110 INFO:teuthology.lock.ops:Updating [trial098.front.sepia.ceph.com]: reset os type and version on server
2026-02-12T08:33:55.110 INFO:teuthology.lock.ops:Updating trial098.front.sepia.ceph.com on lock server
2026-02-12T08:33:55.325 DEBUG:teuthology.provision.fog.trial098:Requesting image trial_centos_9.stream (ID 3)
2026-02-12T08:33:55.394 INFO:teuthology.provision.fog.trial098:Scheduling deploy of centos 9.stream
2026-02-12T08:33:56.247 INFO:teuthology.orchestra.console.trial098:Power off
2026-02-12T08:33:56.247 DEBUG:teuthology.orchestra.console.trial098:pexpect command: ipmitool -H trial098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T08:33:56.288 DEBUG:teuthology.orchestra.console.trial098:power off output: Chassis Power Control: Down/Off
2026-02-12T08:34:00.293 DEBUG:teuthology.orchestra.console.trial098:pexpect command: ipmitool -H trial098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:34:00.402 DEBUG:teuthology.orchestra.console.trial098:check power output: Chassis Power is off
2026-02-12T08:34:00.402 INFO:teuthology.orchestra.console.trial098:Power off completed
2026-02-12T08:34:00.503 INFO:teuthology.orchestra.console.trial098:Power on
2026-02-12T08:34:00.503 DEBUG:teuthology.orchestra.console.trial098:pexpect command: ipmitool -H trial098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-12T08:34:00.544 DEBUG:teuthology.orchestra.console.trial098:power on output: Chassis Power Control: Up/On
2026-02-12T08:34:04.549 DEBUG:teuthology.orchestra.console.trial098:pexpect command: ipmitool -H trial098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:34:04.659 DEBUG:teuthology.orchestra.console.trial098:check power output: Chassis Power is on
2026-02-12T08:34:04.660 INFO:teuthology.orchestra.console.trial098:Power on completed
2026-02-12T08:34:04.760 INFO:teuthology.provision.fog.trial098:Waiting for deploy to finish
2026-02-12T08:36:07.053 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:36:10.141 WARNING:teuthology.provision.fog.trial098:[Errno None] Unable to connect to port 22 on 10.20.193.98
2026-02-12T08:36:16.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:36:19.229 WARNING:teuthology.provision.fog.trial098:[Errno None] Unable to connect to port 22 on 10.20.193.98
2026-02-12T08:36:25.236 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:36:28.317 WARNING:teuthology.provision.fog.trial098:[Errno None] Unable to connect to port 22 on 10.20.193.98
2026-02-12T08:36:34.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:37:10.561 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-12T08:37:10.561 DEBUG:teuthology.orchestra.run.trial098:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-12T08:37:10.722 INFO:teuthology.provision.fog.trial098:Node is ready
2026-02-12T08:37:10.722 DEBUG:teuthology.orchestra.run.trial098:> hostname
2026-02-12T08:37:10.778 INFO:teuthology.orchestra.run.trial098.stdout:trial098.front.sepia.ceph.com
2026-02-12T08:37:10.779 DEBUG:teuthology.orchestra.run.trial098:> grep trial098.front.sepia.ceph.com /etc/hosts
2026-02-12T08:37:10.834 INFO:teuthology.orchestra.run.trial098.stdout:10.20.193.98 trial098.front.sepia.ceph.com trial098
2026-02-12T08:37:10.835 DEBUG:teuthology.orchestra.run.trial098:> sudo hostname trial098
2026-02-12T08:37:10.905 DEBUG:teuthology.orchestra.run.trial098:> sudo sed -i -e 's/trial098.front.sepia.ceph.com/trial098/g' /etc/hosts
2026-02-12T08:37:10.970 DEBUG:teuthology.orchestra.run.trial098:> sudo sed -i -e 's/10.20.193.98/10.20.193.98/g' /etc/hosts
2026-02-12T08:37:11.036 DEBUG:teuthology.orchestra.run.trial098:> sudo sed -i -e 's/trial098.front.sepia.ceph.com/trial098/g' /etc/hostname
2026-02-12T08:37:11.100 DEBUG:teuthology.orchestra.run.trial098:> sudo hostname trial098
2026-02-12T08:37:11.163 DEBUG:teuthology.orchestra.run.trial098:> cat /etc/os-release
2026-02-12T08:37:11.220 INFO:teuthology.orchestra.run.trial098.stdout:NAME="CentOS Stream"
2026-02-12T08:37:11.220 INFO:teuthology.orchestra.run.trial098.stdout:VERSION="9"
2026-02-12T08:37:11.220 INFO:teuthology.orchestra.run.trial098.stdout:ID="centos"
2026-02-12T08:37:11.220 INFO:teuthology.orchestra.run.trial098.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:VERSION_ID="9"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:ANSI_COLOR="0;31"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:37:11.221 INFO:teuthology.orchestra.run.trial098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:37:11.222 INFO:teuthology.provision.fog.trial098:Deploy complete!
2026-02-12T08:37:11.223 DEBUG:teuthology.parallel:result is None
2026-02-12T08:37:11.490 INFO:teuthology.lock.ops:Checking trial098.front.sepia.ceph.com
2026-02-12T08:37:11.490 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-12T08:37:11.579 INFO:teuthology.lock.ops:Updating [trial098.front.sepia.ceph.com]: set os type and version on server
2026-02-12T08:37:11.579 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial098.front.sepia.ceph.com'
2026-02-12T08:37:11.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-12T08:37:11.644 DEBUG:teuthology.orchestra.run.trial098:> true
2026-02-12T08:37:11.713 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial098.front.sepia.ceph.com'
2026-02-12T08:37:11.713 DEBUG:teuthology.orchestra.run.trial098:> uname -m
2026-02-12T08:37:11.768 INFO:teuthology.orchestra.run.trial098.stdout:x86_64
2026-02-12T08:37:11.768 DEBUG:teuthology.orchestra.run.trial098:> cat /etc/os-release
2026-02-12T08:37:11.824 INFO:teuthology.orchestra.run.trial098.stdout:NAME="CentOS Stream"
2026-02-12T08:37:11.824 INFO:teuthology.orchestra.run.trial098.stdout:VERSION="9"
2026-02-12T08:37:11.824 INFO:teuthology.orchestra.run.trial098.stdout:ID="centos"
2026-02-12T08:37:11.824 INFO:teuthology.orchestra.run.trial098.stdout:ID_LIKE="rhel fedora"
2026-02-12T08:37:11.824 INFO:teuthology.orchestra.run.trial098.stdout:VERSION_ID="9"
2026-02-12T08:37:11.824 INFO:teuthology.orchestra.run.trial098.stdout:PLATFORM_ID="platform:el9"
2026-02-12T08:37:11.825 INFO:teuthology.orchestra.run.trial098.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-12T08:37:11.825 INFO:teuthology.orchestra.run.trial098.stdout:ANSI_COLOR="0;31"
2026-02-12T08:37:11.825 INFO:teuthology.orchestra.run.trial098.stdout:LOGO="fedora-logo-icon"
2026-02-12T08:37:11.825 INFO:teuthology.orchestra.run.trial098.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-12T08:37:11.825 INFO:teuthology.orchestra.run.trial098.stdout:HOME_URL="https://centos.org/"
2026-02-12T08:37:11.825 INFO:teuthology.orchestra.run.trial098.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-12T08:37:11.825 INFO:teuthology.orchestra.run.trial098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-12T08:37:11.825 INFO:teuthology.orchestra.run.trial098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-12T08:37:11.826 INFO:teuthology.lock.ops:Updating trial098.front.sepia.ceph.com on lock server
2026-02-12T08:37:11.928 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:37:12.024 INFO:teuthology.dispatcher.supervisor:Running job 41557
2026-02-12T08:37:12.024 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/virtualenv/bin/teuthology -v --owner scheduled_matan@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41557 --name matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial --description crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/sample_fio} -- /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41557/orig.config.yaml
2026-02-12T08:37:12.024 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41557
2026-02-12T08:37:12.024 INFO:teuthology.dispatcher.supervisor:Job PID: 3348902
2026-02-12T08:37:12.024 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-12T08:39:12.120 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:41:12.303 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:43:12.515 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:45:12.725 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:47:12.919 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:49:13.115 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:51:13.312 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:53:13.516 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:55:13.670 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:57:13.875 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:59:14.066 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-12T08:59:14.151 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-12T08:59:14.211 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-12T08:59:14.511 INFO:teuthology.orchestra.console.trial098:Power off
2026-02-12T08:59:14.511 DEBUG:teuthology.orchestra.console.trial098:pexpect command: ipmitool -H trial098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-12T08:59:14.550 DEBUG:teuthology.orchestra.console.trial098:power off output: Chassis Power Control: Down/Off
2026-02-12T08:59:18.555 DEBUG:teuthology.orchestra.console.trial098:pexpect command: ipmitool -H trial098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:59:18.664 DEBUG:teuthology.orchestra.console.trial098:check power output: Chassis Power is on
2026-02-12T08:59:22.669 DEBUG:teuthology.orchestra.console.trial098:pexpect command: ipmitool -H trial098.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-12T08:59:22.779 DEBUG:teuthology.orchestra.console.trial098:check power output: Chassis Power is off
2026-02-12T08:59:22.779 INFO:teuthology.orchestra.console.trial098:Power off completed
2026-02-12T08:59:22.989 INFO:teuthology.lock.ops:unlocked: trial098.front.sepia.ceph.com