2026-02-26T03:01:38.997 INFO:root:teuthology version: 1.2.4.dev11+g6e3d0419c
2026-02-26T03:01:38.998 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:01:39.085 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:01:39.223 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7df9c87b6780>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7df9c803a840>, signals=[15])
2026-02-26T03:01:39.223 INFO:teuthology.lock.ops:Start node 'trial111.front.sepia.ceph.com' reimaging
2026-02-26T03:01:39.223 INFO:teuthology.lock.ops:Updating [trial111.front.sepia.ceph.com]: reset os type and version on server
2026-02-26T03:01:39.223 INFO:teuthology.lock.ops:Updating trial111.front.sepia.ceph.com on lock server
2026-02-26T03:01:39.424 DEBUG:teuthology.provision.fog.trial111:Requesting image trial_centos_9.stream (ID 3)
2026-02-26T03:01:39.478 INFO:teuthology.provision.fog.trial111:Scheduling deploy of centos 9.stream
2026-02-26T03:01:40.989 INFO:teuthology.orchestra.console.trial111:Power off
2026-02-26T03:01:40.989 DEBUG:teuthology.orchestra.console.trial111:pexpect command: ipmitool -H trial111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-26T03:01:41.027 DEBUG:teuthology.orchestra.console.trial111:power off output: Chassis Power Control: Down/Off
2026-02-26T03:01:45.032 DEBUG:teuthology.orchestra.console.trial111:pexpect command: ipmitool -H trial111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-26T03:01:45.139 DEBUG:teuthology.orchestra.console.trial111:check power output: Chassis Power is off
2026-02-26T03:01:45.139 INFO:teuthology.orchestra.console.trial111:Power off completed
2026-02-26T03:01:45.240 INFO:teuthology.orchestra.console.trial111:Power on
2026-02-26T03:01:45.240 DEBUG:teuthology.orchestra.console.trial111:pexpect command: ipmitool -H trial111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on
2026-02-26T03:01:45.287 DEBUG:teuthology.orchestra.console.trial111:power on output: Chassis Power Control: Up/On
2026-02-26T03:01:49.291 DEBUG:teuthology.orchestra.console.trial111:pexpect command: ipmitool -H trial111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-26T03:01:49.402 DEBUG:teuthology.orchestra.console.trial111:check power output: Chassis Power is on
2026-02-26T03:01:49.402 INFO:teuthology.orchestra.console.trial111:Power on completed
2026-02-26T03:01:49.502 INFO:teuthology.provision.fog.trial111:Waiting for deploy to finish
2026-02-26T03:03:53.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-26T03:03:56.160 WARNING:teuthology.provision.fog.trial111:[Errno None] Unable to connect to port 22 on 10.20.193.111
2026-02-26T03:04:02.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-26T03:04:05.248 WARNING:teuthology.provision.fog.trial111:[Errno None] Unable to connect to port 22 on 10.20.193.111
2026-02-26T03:04:11.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-26T03:04:14.336 WARNING:teuthology.provision.fog.trial111:[Errno None] Unable to connect to port 22 on 10.20.193.111
2026-02-26T03:04:20.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-26T03:04:23.424 WARNING:teuthology.provision.fog.trial111:[Errno None] Unable to connect to port 22 on 10.20.193.111
2026-02-26T03:04:29.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-26T03:05:05.274 INFO:teuthology.orchestra.run:Running command with timeout 600
2026-02-26T03:05:05.274 DEBUG:teuthology.orchestra.run.trial111:> while [ ! -e '/.cephlab_hostname_set' ]; do sleep 5; done
2026-02-26T03:05:05.427 INFO:teuthology.provision.fog.trial111:Node is ready
2026-02-26T03:05:05.427 DEBUG:teuthology.orchestra.run.trial111:> hostname
2026-02-26T03:05:05.481 INFO:teuthology.orchestra.run.trial111.stdout:trial111.front.sepia.ceph.com
2026-02-26T03:05:05.482 DEBUG:teuthology.orchestra.run.trial111:> grep trial111.front.sepia.ceph.com /etc/hosts
2026-02-26T03:05:05.538 INFO:teuthology.orchestra.run.trial111.stdout:10.20.193.111 trial111.front.sepia.ceph.com trial111
2026-02-26T03:05:05.538 DEBUG:teuthology.orchestra.run.trial111:> sudo hostname trial111
2026-02-26T03:05:05.608 DEBUG:teuthology.orchestra.run.trial111:> sudo sed -i -e 's/trial111.front.sepia.ceph.com/trial111/g' /etc/hosts
2026-02-26T03:05:05.674 DEBUG:teuthology.orchestra.run.trial111:> sudo sed -i -e 's/10.20.193.111/10.20.193.111/g' /etc/hosts
2026-02-26T03:05:05.740 DEBUG:teuthology.orchestra.run.trial111:> sudo sed -i -e 's/trial111.front.sepia.ceph.com/trial111/g' /etc/hostname
2026-02-26T03:05:05.804 DEBUG:teuthology.orchestra.run.trial111:> sudo hostname trial111
2026-02-26T03:05:05.867 DEBUG:teuthology.orchestra.run.trial111:> cat /etc/os-release
2026-02-26T03:05:05.923 INFO:teuthology.orchestra.run.trial111.stdout:NAME="CentOS Stream"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:VERSION="9"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:ID="centos"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:ID_LIKE="rhel fedora"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:VERSION_ID="9"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:PLATFORM_ID="platform:el9"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:ANSI_COLOR="0;31"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:LOGO="fedora-logo-icon"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-26T03:05:05.924 INFO:teuthology.orchestra.run.trial111.stdout:HOME_URL="https://centos.org/"
2026-02-26T03:05:05.925 INFO:teuthology.orchestra.run.trial111.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-26T03:05:05.925 INFO:teuthology.orchestra.run.trial111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-26T03:05:05.925 INFO:teuthology.orchestra.run.trial111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-26T03:05:05.926 INFO:teuthology.provision.fog.trial111:Deploy complete!
2026-02-26T03:05:05.926 DEBUG:teuthology.parallel:result is None
2026-02-26T03:05:06.220 INFO:teuthology.lock.ops:Checking trial111.front.sepia.ceph.com
2026-02-26T03:05:06.220 INFO:teuthology.lock.ops:New key found. Updating...
2026-02-26T03:05:06.284 INFO:teuthology.lock.ops:Updating [trial111.front.sepia.ceph.com]: set os type and version on server
2026-02-26T03:05:06.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial111.front.sepia.ceph.com'
2026-02-26T03:05:06.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-26T03:05:06.345 DEBUG:teuthology.orchestra.run.trial111:> true
2026-02-26T03:05:06.419 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial111.front.sepia.ceph.com'
2026-02-26T03:05:06.419 DEBUG:teuthology.orchestra.run.trial111:> uname -m
2026-02-26T03:05:06.475 INFO:teuthology.orchestra.run.trial111.stdout:x86_64
2026-02-26T03:05:06.475 DEBUG:teuthology.orchestra.run.trial111:> cat /etc/os-release
2026-02-26T03:05:06.530 INFO:teuthology.orchestra.run.trial111.stdout:NAME="CentOS Stream"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:VERSION="9"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:ID="centos"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:ID_LIKE="rhel fedora"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:VERSION_ID="9"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:PLATFORM_ID="platform:el9"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:ANSI_COLOR="0;31"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:LOGO="fedora-logo-icon"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:HOME_URL="https://centos.org/"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-26T03:05:06.531 INFO:teuthology.orchestra.run.trial111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-26T03:05:06.531 INFO:teuthology.lock.ops:Updating trial111.front.sepia.ceph.com on lock server
2026-02-26T03:05:06.591 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:05:06.680 INFO:teuthology.dispatcher.supervisor:Running job 71169
2026-02-26T03:05:06.680 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_6e3d0419ccca0e75a03e08feb708946e198236f4/virtualenv/bin/teuthology -v --owner scheduled_shraddhaag@soko04.front.sepia.ceph.com --archive /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71169 --name shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial --description crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_rand_read} -- /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71169/orig.config.yaml
2026-02-26T03:05:06.681 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/mnt/teuthology/shraddhaag-2026-02-26_02:22:56-crimson-rados-main-distro-debug-trial/71169
2026-02-26T03:05:06.681 INFO:teuthology.dispatcher.supervisor:Job PID: 892839
2026-02-26T03:05:06.681 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2026-02-26T03:07:06.707 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:09:06.867 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:11:07.075 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:13:07.168 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:15:07.371 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:17:07.576 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:19:07.832 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:21:08.034 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:23:08.233 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:25:08.419 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:27:08.612 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:29:08.816 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:31:08.945 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:33:09.133 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:35:09.310 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-26T03:35:09.380 INFO:teuthology.dispatcher.supervisor:Success!
2026-02-26T03:35:09.439 INFO:teuthology.dispatcher.supervisor:Unlocking machines...
2026-02-26T03:35:09.728 INFO:teuthology.orchestra.console.trial111:Power off
2026-02-26T03:35:09.728 DEBUG:teuthology.orchestra.console.trial111:pexpect command: ipmitool -H trial111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off
2026-02-26T03:35:09.767 DEBUG:teuthology.orchestra.console.trial111:power off output: Chassis Power Control: Down/Off
2026-02-26T03:35:13.771 DEBUG:teuthology.orchestra.console.trial111:pexpect command: ipmitool -H trial111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-26T03:35:13.880 DEBUG:teuthology.orchestra.console.trial111:check power output: Chassis Power is on
2026-02-26T03:35:17.880 DEBUG:teuthology.orchestra.console.trial111:pexpect command: ipmitool -H trial111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status
2026-02-26T03:35:17.990 DEBUG:teuthology.orchestra.console.trial111:check power output: Chassis Power is off
2026-02-26T03:35:17.991 INFO:teuthology.orchestra.console.trial111:Power off completed
2026-02-26T03:35:18.154 INFO:teuthology.lock.ops:unlocked: trial111.front.sepia.ceph.com