2026-02-12T09:02:00.220 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062 2026-02-12T09:02:00.235 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T09:02:00.321 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41601 branch: main description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/radosbench_omap_write} email: null first_in_suite: false flavor: debug job_id: '41601' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: 2401a109e2ac85dfa20fc57328da663aa754c875 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 2401a109e2ac85dfa20fc57328da663aa754c875 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c owner: scheduled_matan@soko04.front.sepia.ceph.com priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 9353 sha1: 2401a109e2ac85dfa20fc57328da663aa754c875 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c targets: trial057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXdTLia5nq7hSbgAURcBVolnepOvqJYXd+BQz8BQ6nuVui+R6GjDga6XdvVAe3u4su4oGQCyCGLNLA2gZytTuw= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - radosbench: clients: - client.0 objectsize: 4096 size: 4096 time: 300 write-omap: true teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: c433f1062990a0488dc29a553589bc609a460691 timestamp: 2026-02-09_15:50:17 tube: trial user: matan verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.2161924 2026-02-12T09:02:00.322 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c/qa; will attempt to use it 2026-02-12T09:02:00.322 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c/qa/tasks 2026-02-12T09:02:00.322 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-12T09:02:00.324 INFO:teuthology.task.internal:Checking packages... 2026-02-12T09:02:00.324 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '2401a109e2ac85dfa20fc57328da663aa754c875' 2026-02-12T09:02:00.324 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-12T09:02:00.324 INFO:teuthology.packaging:ref: None 2026-02-12T09:02:00.324 INFO:teuthology.packaging:tag: None 2026-02-12T09:02:00.324 INFO:teuthology.packaging:branch: main 2026-02-12T09:02:00.324 INFO:teuthology.packaging:sha1: 2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:02:00.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=main 2026-02-12T09:02:00.486 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5271.gdbd7cb72 2026-02-12T09:02:00.487 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-12T09:02:00.499 INFO:teuthology.task.internal:no buildpackages task found 2026-02-12T09:02:00.499 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-12T09:02:00.513 INFO:teuthology.task.internal:Saving configuration 2026-02-12T09:02:00.532 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-12T09:02:00.544 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-12T09:02:00.630 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial057.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41601', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 08:58:47.976112', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXdTLia5nq7hSbgAURcBVolnepOvqJYXd+BQz8BQ6nuVui+R6GjDga6XdvVAe3u4su4oGQCyCGLNLA2gZytTuw='} 2026-02-12T09:02:00.630 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-12T09:02:00.641 INFO:teuthology.task.internal:roles: ubuntu@trial057.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-12T09:02:00.641 INFO:teuthology.run_tasks:Running task console_log... 2026-02-12T09:02:00.732 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x72710af05580>, signals=[15]) 2026-02-12T09:02:00.732 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-12T09:02:00.744 INFO:teuthology.task.internal:Opening connections... 2026-02-12T09:02:00.744 DEBUG:teuthology.task.internal:connecting to ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:02:00.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T09:02:00.807 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-12T09:02:00.819 DEBUG:teuthology.orchestra.run.trial057:> uname -m 2026-02-12T09:02:00.887 INFO:teuthology.orchestra.run.trial057.stdout:x86_64 2026-02-12T09:02:00.888 DEBUG:teuthology.orchestra.run.trial057:> cat /etc/os-release 2026-02-12T09:02:00.943 INFO:teuthology.orchestra.run.trial057.stdout:NAME="CentOS Stream" 2026-02-12T09:02:00.943 INFO:teuthology.orchestra.run.trial057.stdout:VERSION="9" 2026-02-12T09:02:00.943 INFO:teuthology.orchestra.run.trial057.stdout:ID="centos" 2026-02-12T09:02:00.943 INFO:teuthology.orchestra.run.trial057.stdout:ID_LIKE="rhel fedora" 2026-02-12T09:02:00.943 INFO:teuthology.orchestra.run.trial057.stdout:VERSION_ID="9" 2026-02-12T09:02:00.943 INFO:teuthology.orchestra.run.trial057.stdout:PLATFORM_ID="platform:el9" 2026-02-12T09:02:00.943 INFO:teuthology.orchestra.run.trial057.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T09:02:00.944 INFO:teuthology.orchestra.run.trial057.stdout:ANSI_COLOR="0;31" 2026-02-12T09:02:00.944 INFO:teuthology.orchestra.run.trial057.stdout:LOGO="fedora-logo-icon" 2026-02-12T09:02:00.944 INFO:teuthology.orchestra.run.trial057.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T09:02:00.944 INFO:teuthology.orchestra.run.trial057.stdout:HOME_URL="https://centos.org/" 2026-02-12T09:02:00.944 INFO:teuthology.orchestra.run.trial057.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T09:02:00.944 INFO:teuthology.orchestra.run.trial057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T09:02:00.944 INFO:teuthology.orchestra.run.trial057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T09:02:00.945 INFO:teuthology.lock.ops:Updating trial057.front.sepia.ceph.com on lock server 2026-02-12T09:02:01.041 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-12T09:02:01.054 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-12T09:02:01.066 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-12T09:02:01.066 DEBUG:teuthology.orchestra.run.trial057:> test '!' -e /home/ubuntu/cephtest 2026-02-12T09:02:01.081 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-12T09:02:01.093 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-12T09:02:01.093 DEBUG:teuthology.orchestra.run.trial057:> test -z $(ls -A /var/lib/ceph) 2026-02-12T09:02:01.138 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-12T09:02:01.264 INFO:teuthology.run_tasks:Running task kernel... 2026-02-12T09:02:01.287 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-12T09:02:01.287 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-12T09:02:01.287 DEBUG:teuthology.orchestra.run.trial057:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T09:02:01.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T09:02:01.303 DEBUG:teuthology.orchestra.run.trial057:> uname -r 2026-02-12T09:02:01.359 INFO:teuthology.orchestra.run.trial057.stdout:5.14.0-645.el9.x86_64 2026-02-12T09:02:01.359 INFO:teuthology.task.kernel:Running kernel on trial057: 5.14.0-645.el9.x86_64 2026-02-12T09:02:01.359 DEBUG:teuthology.orchestra.run.trial057:> sudo yum install -y kernel 2026-02-12T09:02:09.883 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-12T09:02:12.915 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-12T09:02:16.425 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 7.8 MB 00:01 2026-02-12T09:02:18.203 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 20 kB 00:01 2026-02-12T09:02:18.898 INFO:teuthology.orchestra.run.trial057.stdout:Extra Packages for Enterprise Linux 33 MB/s | 20 MB 00:00 2026-02-12T09:02:22.080 INFO:teuthology.orchestra.run.trial057.stdout:lab-extras 1.6 MB/s | 50 kB 00:00 2026-02-12T09:02:23.153 INFO:teuthology.orchestra.run.trial057.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T09:02:23.173 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:02:23.174 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:02:23.174 INFO:teuthology.orchestra.run.trial057.stdout: Package Architecture Version Repository Size 2026-02-12T09:02:23.174 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:02:23.174 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:02:23.174 INFO:teuthology.orchestra.run.trial057.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T09:02:23.174 INFO:teuthology.orchestra.run.trial057.stdout:Installing dependencies: 2026-02-12T09:02:23.174 INFO:teuthology.orchestra.run.trial057.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T09:02:23.174 INFO:teuthology.orchestra.run.trial057.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout:Install 4 Packages 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 88 M 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 135 M 2026-02-12T09:02:23.175 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:02:26.068 INFO:teuthology.orchestra.run.trial057.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 340 kB/s | 768 kB 00:02 2026-02-12T09:02:43.132 INFO:teuthology.orchestra.run.trial057.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 903 kB/s | 17 MB 00:19 2026-02-12T09:03:00.260 INFO:teuthology.orchestra.run.trial057.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 922 kB/s | 31 MB 00:34 2026-02-12T09:03:08.467 INFO:teuthology.orchestra.run.trial057.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 908 kB/s | 40 MB 00:44 2026-02-12T09:03:08.469 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:03:08.469 INFO:teuthology.orchestra.run.trial057.stdout:Total 1.9 MB/s | 88 MB 00:45 2026-02-12T09:03:08.658 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:03:08.861 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:03:08.861 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:03:09.209 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:03:09.209 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:03:10.303 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:03:10.586 INFO:teuthology.orchestra.run.trial057.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T09:03:10.605 INFO:teuthology.orchestra.run.trial057.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T09:03:12.370 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T09:03:12.399 INFO:teuthology.orchestra.run.trial057.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T09:03:15.424 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T09:03:15.430 INFO:teuthology.orchestra.run.trial057.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:03:18.528 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:03:48.899 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:03:48.920 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:03:48.984 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:03:48.985 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T09:03:48.985 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T09:03:48.985 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T09:03:49.037 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:03:49.037 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:03:49.037 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:03:49.038 INFO:teuthology.orchestra.run.trial057.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T09:03:49.038 INFO:teuthology.orchestra.run.trial057.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T09:03:49.038 INFO:teuthology.orchestra.run.trial057.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T09:03:49.038 INFO:teuthology.orchestra.run.trial057.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T09:03:49.038 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:03:49.038 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:03:49.093 DEBUG:teuthology.orchestra.run.trial057:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T09:03:49.118 INFO:teuthology.orchestra.run.trial057.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T09:03:49.119 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T09:03:49.119 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T09:03:49.119 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T09:03:49.120 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T09:03:49.120 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-12T09:03:49.120 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial057.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T09:03:49.120 DEBUG:teuthology.orchestra.run.trial057:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T09:03:49.183 INFO:teuthology.orchestra.run.trial057.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T09:03:49.184 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T09:03:49.184 DEBUG:teuthology.orchestra.run.trial057:> sudo rpm -qi grub2-tools 2026-02-12T09:03:49.254 INFO:teuthology.orchestra.run.trial057.stdout:Name : grub2-tools 2026-02-12T09:03:49.254 INFO:teuthology.orchestra.run.trial057.stdout:Epoch : 1 2026-02-12T09:03:49.254 INFO:teuthology.orchestra.run.trial057.stdout:Version : 2.06 2026-02-12T09:03:49.254 INFO:teuthology.orchestra.run.trial057.stdout:Release : 118.el9 2026-02-12T09:03:49.254 INFO:teuthology.orchestra.run.trial057.stdout:Architecture: x86_64 2026-02-12T09:03:49.254 INFO:teuthology.orchestra.run.trial057.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Group : Unspecified 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Size : 8393909 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:License : GPLv3+ 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Packager : builder@centos.org 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Vendor : CentOS 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Summary : Support tools for GRUB. 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout:Description : 2026-02-12T09:03:49.255 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:03:49.256 INFO:teuthology.orchestra.run.trial057.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T09:03:49.256 INFO:teuthology.orchestra.run.trial057.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T09:03:49.256 INFO:teuthology.orchestra.run.trial057.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T09:03:49.256 INFO:teuthology.orchestra.run.trial057.stdout:hardware devices. 2026-02-12T09:03:49.256 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:03:49.256 INFO:teuthology.orchestra.run.trial057.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T09:03:49.257 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T09:03:49.257 INFO:teuthology.task.kernel:Updating grub on trial057 to boot 5.14.0-677.el9.x86_64 2026-02-12T09:03:49.257 DEBUG:teuthology.orchestra.run.trial057:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T09:03:49.321 INFO:teuthology.orchestra.run.trial057.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T09:03:49.323 DEBUG:teuthology.orchestra.run.trial057:> sudo command -v grubby && echo yes || echo no 2026-02-12T09:03:49.386 INFO:teuthology.orchestra.run.trial057.stdout:/sbin/grubby 2026-02-12T09:03:49.387 INFO:teuthology.orchestra.run.trial057.stdout:yes 2026-02-12T09:03:49.387 DEBUG:teuthology.orchestra.run.trial057:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T09:03:49.452 INFO:teuthology.orchestra.run.trial057.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T09:03:49.453 DEBUG:teuthology.orchestra.run.trial057:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T09:03:49.526 DEBUG:teuthology.orchestra.run.trial057:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T09:03:49.736 INFO:teuthology.orchestra.run.trial057.stderr:Generating grub configuration file ... 2026-02-12T09:03:49.797 INFO:teuthology.orchestra.run.trial057.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T09:03:49.804 INFO:teuthology.orchestra.run.trial057.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T09:03:50.057 INFO:teuthology.orchestra.run.trial057.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T09:03:50.064 INFO:teuthology.orchestra.run.trial057.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T09:03:50.096 INFO:teuthology.orchestra.run.trial057.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T09:03:50.103 INFO:teuthology.orchestra.run.trial057.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T09:03:50.126 INFO:teuthology.orchestra.run.trial057.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T09:03:50.133 INFO:teuthology.orchestra.run.trial057.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T09:03:50.214 INFO:teuthology.orchestra.run.trial057.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T09:03:50.221 INFO:teuthology.orchestra.run.trial057.stderr:done 2026-02-12T09:03:50.223 DEBUG:teuthology.orchestra.run.trial057:> test -d /sys/firmware/efi 2026-02-12T09:03:50.240 DEBUG:teuthology.orchestra.run.trial057:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T09:03:50.304 INFO:teuthology.orchestra.run.trial057.stdout:/boot/efi/EFI/BOOT 2026-02-12T09:03:50.304 INFO:teuthology.orchestra.run.trial057.stdout:/boot/efi/EFI/centos 2026-02-12T09:03:50.306 DEBUG:teuthology.orchestra.run.trial057:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T09:03:50.371 INFO:teuthology.orchestra.run.trial057.stdout:no 2026-02-12T09:03:50.372 DEBUG:teuthology.orchestra.run.trial057:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T09:03:50.435 INFO:teuthology.orchestra.run.trial057.stdout:yes 2026-02-12T09:03:50.436 DEBUG:teuthology.orchestra.run.trial057:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T09:03:50.499 DEBUG:teuthology.orchestra.run.trial057:> sudo shutdown -r now 2026-02-12T09:03:50.574 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-12T09:04:20.572 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-12T09:04:20.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial057.front.sepia.ceph.com' 2026-02-12T09:04:20.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T09:05:20.635 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T09:05:29.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial057.front.sepia.ceph.com' 2026-02-12T09:05:29.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T09:05:29.782 DEBUG:teuthology.orchestra.run.trial057:> true 2026-02-12T09:05:29.939 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial057.front.sepia.ceph.com' 2026-02-12T09:05:29.939 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-12T09:05:29.939 DEBUG:teuthology.orchestra.run.trial057:> uname -r 2026-02-12T09:05:29.995 INFO:teuthology.orchestra.run.trial057.stdout:5.14.0-677.el9.x86_64 2026-02-12T09:05:29.995 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-12T09:05:29.995 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T09:05:29.996 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-12T09:05:30.997 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-12T09:05:30.997 DEBUG:teuthology.orchestra.run.trial057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T09:05:31.027 INFO:teuthology.orchestra.run.trial057.stdout:ttyS1 2026-02-12T09:05:31.051 DEBUG:teuthology.parallel:result is None 2026-02-12T09:05:31.051 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-12T09:05:31.062 INFO:teuthology.task.internal:Creating test directory... 2026-02-12T09:05:31.062 DEBUG:teuthology.orchestra.run.trial057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-12T09:05:31.107 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-12T09:05:31.118 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-12T09:05:31.132 INFO:teuthology.task.internal:Creating archive directory... 2026-02-12T09:05:31.132 DEBUG:teuthology.orchestra.run.trial057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-12T09:05:31.172 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-12T09:05:31.183 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-12T09:05:31.183 DEBUG:teuthology.orchestra.run.trial057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-12T09:05:31.229 INFO:teuthology.orchestra.run.trial057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T09:05:31.238 INFO:teuthology.orchestra.run.trial057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T09:05:31.240 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-12T09:05:31.252 INFO:teuthology.task.internal:Configuring sudo... 2026-02-12T09:05:31.252 DEBUG:teuthology.orchestra.run.trial057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-12T09:05:31.303 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-12T09:05:31.319 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-12T09:05:31.320 DEBUG:teuthology.orchestra.run.trial057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-12T09:05:31.361 DEBUG:teuthology.orchestra.run.trial057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T09:05:31.527 DEBUG:teuthology.orchestra.run.trial057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T09:05:31.552 DEBUG:teuthology.orchestra.run.trial057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T09:05:31.700 DEBUG:teuthology.orchestra.run.trial057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T09:05:31.725 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:05:31.725 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-12T09:05:31.789 DEBUG:teuthology.orchestra.run.trial057:> sudo service rsyslog restart 2026-02-12T09:05:31.856 INFO:teuthology.orchestra.run.trial057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T09:05:32.263 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-12T09:05:32.276 INFO:teuthology.task.internal:Starting timer... 2026-02-12T09:05:32.276 INFO:teuthology.run_tasks:Running task pcp... 2026-02-12T09:05:32.298 INFO:teuthology.run_tasks:Running task selinux... 2026-02-12T09:05:32.316 DEBUG:teuthology.orchestra.run.trial057:> sudo service auditd rotate 2026-02-12T09:05:32.344 INFO:teuthology.orchestra.run.trial057.stdout:Rotating logs: 2026-02-12T09:05:32.346 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-12T09:05:32.346 DEBUG:teuthology.orchestra.run.trial057:> /usr/sbin/getenforce 2026-02-12T09:05:32.402 INFO:teuthology.orchestra.run.trial057.stdout:Permissive 2026-02-12T09:05:32.402 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial057.front.sepia.ceph.com': 'permissive'} 2026-02-12T09:05:32.402 DEBUG:teuthology.orchestra.run.trial057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T09:05:32.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T09:05:32.466 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-12T09:05:32.466 DEBUG:teuthology.orchestra.run.trial057:> sudo /usr/sbin/setenforce permissive 2026-02-12T09:05:32.529 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-12T09:05:32.544 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-12T09:05:32.548 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-12T09:05:32.666 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-12T09:05:32.677 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-12T09:05:32.679 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial057.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-12T09:06:38.362 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial057.front.sepia.ceph.com')] 2026-02-12T09:06:38.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial057.front.sepia.ceph.com' 2026-02-12T09:06:38.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T09:06:38.428 DEBUG:teuthology.orchestra.run.trial057:> true 2026-02-12T09:06:38.509 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial057.front.sepia.ceph.com' 2026-02-12T09:06:38.510 INFO:teuthology.run_tasks:Running task clock... 2026-02-12T09:06:38.522 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-12T09:06:38.522 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-12T09:06:38.522 DEBUG:teuthology.orchestra.run.trial057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T09:06:38.575 INFO:teuthology.orchestra.run.trial057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-12T09:06:38.586 INFO:teuthology.orchestra.run.trial057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-12T09:06:38.606 INFO:teuthology.orchestra.run.trial057.stderr:sudo: ntpd: command not found 2026-02-12T09:06:38.616 INFO:teuthology.orchestra.run.trial057.stdout:506 Cannot talk to daemon 2026-02-12T09:06:38.627 INFO:teuthology.orchestra.run.trial057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-12T09:06:38.639 INFO:teuthology.orchestra.run.trial057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-12T09:06:38.693 INFO:teuthology.orchestra.run.trial057.stderr:bash: line 1: ntpq: command not found 2026-02-12T09:06:38.695 INFO:teuthology.orchestra.run.trial057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T09:06:38.695 INFO:teuthology.orchestra.run.trial057.stdout:=============================================================================== 2026-02-12T09:06:38.696 INFO:teuthology.run_tasks:Running task install... 2026-02-12T09:06:38.710 DEBUG:teuthology.task.install:project ceph 2026-02-12T09:06:38.711 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}} 2026-02-12T09:06:38.711 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'} 2026-02-12T09:06:38.711 INFO:teuthology.task.install:Using flavor: debug 2026-02-12T09:06:38.721 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T09:06:38.721 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-12T09:06:38.721 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875', 'tag': None, 'wait_for_package': False} 2026-02-12T09:06:38.722 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:06:38.896 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/ 2026-02-12T09:06:38.896 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5161.g2401a109 2026-02-12T09:06:39.033 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T09:06:39.033 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:06:39.033 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T09:06:39.060 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-12T09:06:39.060 DEBUG:teuthology.orchestra.run.trial057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2401a109e2ac85dfa20fc57328da663aa754c875/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T09:06:39.125 DEBUG:teuthology.orchestra.run.trial057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T09:06:39.196 DEBUG:teuthology.orchestra.run.trial057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T09:06:39.259 INFO:teuthology.orchestra.run.trial057.stdout:check_obsoletes = 1 2026-02-12T09:06:39.261 DEBUG:teuthology.orchestra.run.trial057:> sudo yum clean all 2026-02-12T09:06:39.410 INFO:teuthology.orchestra.run.trial057.stdout:41 files removed 2026-02-12T09:06:39.424 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-radosgw 2026-02-12T09:06:39.698 INFO:teuthology.orchestra.run.trial057.stdout:ceph packages for x86_64 738 kB/s | 94 kB 00:00 2026-02-12T09:06:39.839 INFO:teuthology.orchestra.run.trial057.stdout:ceph noarch packages 210 kB/s | 24 kB 00:00 2026-02-12T09:06:39.969 INFO:teuthology.orchestra.run.trial057.stdout:ceph source packages 19 kB/s | 2.1 kB 00:00 2026-02-12T09:06:41.078 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.9 MB 00:01 2026-02-12T09:06:43.270 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - AppStream 16 MB/s | 27 MB 00:01 2026-02-12T09:06:46.144 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - CRB 17 MB/s | 7.8 MB 00:00 2026-02-12T09:06:47.205 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 20 kB 00:00 2026-02-12T09:06:48.439 INFO:teuthology.orchestra.run.trial057.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2026-02-12T09:06:51.729 INFO:teuthology.orchestra.run.trial057.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-12T09:06:53.016 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:06:53.018 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:06:53.018 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repository Size 2026-02-12T09:06:53.018 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:06:53.018 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:06:53.018 INFO:teuthology.orchestra.run.trial057.stdout: ceph-radosgw x86_64 2:20.3.0-5161.g2401a109.el9 ceph 11 M 2026-02-12T09:06:53.018 INFO:teuthology.orchestra.run.trial057.stdout:Upgrading: 2026-02-12T09:06:53.018 INFO:teuthology.orchestra.run.trial057.stdout: librados2 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 3.3 M 2026-02-12T09:06:53.018 INFO:teuthology.orchestra.run.trial057.stdout: librbd1 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 2.7 M 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout:Installing dependencies: 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: ceph-base x86_64 2:20.3.0-5161.g2401a109.el9 ceph 5.1 M 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: ceph-common x86_64 2:20.3.0-5161.g2401a109.el9 ceph 21 M 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: ceph-selinux x86_64 2:20.3.0-5161.g2401a109.el9 ceph 25 k 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs2 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 860 k 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-12T09:06:53.019 INFO:teuthology.orchestra.run.trial057.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: libradosstriper1 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 263 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: librgw2 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 5.6 M 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: python3-ceph-argparse x86_64 2:20.3.0-5161.g2401a109.el9 ceph 45 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: python3-ceph-common x86_64 2:20.3.0-5161.g2401a109.el9 ceph 184 k 2026-02-12T09:06:53.020 INFO:teuthology.orchestra.run.trial057.stdout: python3-cephfs x86_64 2:20.3.0-5161.g2401a109.el9 ceph 204 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: python3-rados x86_64 2:20.3.0-5161.g2401a109.el9 ceph 323 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: python3-rbd x86_64 2:20.3.0-5161.g2401a109.el9 ceph 305 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: python3-rgw x86_64 2:20.3.0-5161.g2401a109.el9 ceph 100 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout:Installing weak dependencies: 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-daemon x86_64 2:20.3.0-5161.g2401a109.el9 ceph 35 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 24 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-12T09:06:53.021 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:06:53.022 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:06:53.022 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:06:53.022 INFO:teuthology.orchestra.run.trial057.stdout:Install 35 Packages 2026-02-12T09:06:53.022 INFO:teuthology.orchestra.run.trial057.stdout:Upgrade 2 Packages 2026-02-12T09:06:53.022 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:06:53.023 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 61 M 2026-02-12T09:06:53.023 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:06:53.586 INFO:teuthology.orchestra.run.trial057.stdout:(1/37): ceph-base-20.3.0-5161.g2401a109.el9.x86 15 MB/s | 5.1 MB 00:00 2026-02-12T09:06:53.625 INFO:teuthology.orchestra.run.trial057.stdout:(2/37): ceph-selinux-20.3.0-5161.g2401a109.el9. 649 kB/s | 25 kB 00:00 2026-02-12T09:06:53.665 INFO:teuthology.orchestra.run.trial057.stdout:(3/37): libcephfs-daemon-20.3.0-5161.g2401a109. 878 kB/s | 35 kB 00:00 2026-02-12T09:06:53.707 INFO:teuthology.orchestra.run.trial057.stdout:(4/37): libcephfs-proxy2-20.3.0-5161.g2401a109. 576 kB/s | 24 kB 00:00 2026-02-12T09:06:53.766 INFO:teuthology.orchestra.run.trial057.stdout:(5/37): libcephfs2-20.3.0-5161.g2401a109.el9.x8 14 MB/s | 860 kB 00:00 2026-02-12T09:06:53.816 INFO:teuthology.orchestra.run.trial057.stdout:(6/37): libradosstriper1-20.3.0-5161.g2401a109. 5.1 MB/s | 263 kB 00:00 2026-02-12T09:06:53.911 INFO:teuthology.orchestra.run.trial057.stdout:(7/37): ceph-radosgw-20.3.0-5161.g2401a109.el9. 17 MB/s | 11 MB 00:00 2026-02-12T09:06:53.961 INFO:teuthology.orchestra.run.trial057.stdout:(8/37): python3-ceph-argparse-20.3.0-5161.g2401 911 kB/s | 45 kB 00:00 2026-02-12T09:06:54.014 INFO:teuthology.orchestra.run.trial057.stdout:(9/37): python3-ceph-common-20.3.0-5161.g2401a1 3.4 MB/s | 184 kB 00:00 2026-02-12T09:06:54.069 INFO:teuthology.orchestra.run.trial057.stdout:(10/37): python3-cephfs-20.3.0-5161.g2401a109.e 3.6 MB/s | 204 kB 00:00 2026-02-12T09:06:54.108 INFO:teuthology.orchestra.run.trial057.stdout:(11/37): librgw2-20.3.0-5161.g2401a109.el9.x86_ 19 MB/s | 5.6 MB 00:00 2026-02-12T09:06:54.125 INFO:teuthology.orchestra.run.trial057.stdout:(12/37): python3-rados-20.3.0-5161.g2401a109.el 5.7 MB/s | 323 kB 00:00 2026-02-12T09:06:54.152 INFO:teuthology.orchestra.run.trial057.stdout:(13/37): python3-rbd-20.3.0-5161.g2401a109.el9. 6.7 MB/s | 305 kB 00:00 2026-02-12T09:06:54.166 INFO:teuthology.orchestra.run.trial057.stdout:(14/37): python3-rgw-20.3.0-5161.g2401a109.el9. 2.4 MB/s | 100 kB 00:00 2026-02-12T09:06:54.225 INFO:teuthology.orchestra.run.trial057.stdout:(15/37): ceph-common-20.3.0-5161.g2401a109.el9. 21 MB/s | 21 MB 00:00 2026-02-12T09:06:54.225 INFO:teuthology.orchestra.run.trial057.stdout:(16/37): mailcap-2.1.49-5.el9.noarch.rpm 454 kB/s | 33 kB 00:00 2026-02-12T09:06:54.292 INFO:teuthology.orchestra.run.trial057.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 1.5 MB/s | 104 kB 00:00 2026-02-12T09:06:54.311 INFO:teuthology.orchestra.run.trial057.stdout:(18/37): pciutils-3.7.0-7.el9.x86_64.rpm 1.1 MB/s | 93 kB 00:00 2026-02-12T09:06:54.333 INFO:teuthology.orchestra.run.trial057.stdout:(19/37): libnbd-1.20.3-4.el9.x86_64.rpm 3.9 MB/s | 164 kB 00:00 2026-02-12T09:06:54.343 INFO:teuthology.orchestra.run.trial057.stdout:(20/37): nvme-cli-2.13-1.el9.x86_64.rpm 6.2 MB/s | 1.1 MB 00:00 2026-02-12T09:06:54.354 INFO:teuthology.orchestra.run.trial057.stdout:(21/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.1 MB/s | 45 kB 00:00 2026-02-12T09:06:54.369 INFO:teuthology.orchestra.run.trial057.stdout:(22/37): librdkafka-1.6.1-102.el9.x86_64.rpm 25 MB/s | 662 kB 00:00 2026-02-12T09:06:54.376 INFO:teuthology.orchestra.run.trial057.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 13 MB/s | 292 kB 00:00 2026-02-12T09:06:54.378 INFO:teuthology.orchestra.run.trial057.stdout:(24/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2026-02-12T09:06:54.394 INFO:teuthology.orchestra.run.trial057.stdout:(25/37): qatlib-25.08.0-2.el9.x86_64.rpm 9.4 MB/s | 240 kB 00:00 2026-02-12T09:06:54.398 INFO:teuthology.orchestra.run.trial057.stdout:(26/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.8 MB/s | 37 kB 00:00 2026-02-12T09:06:54.402 INFO:teuthology.orchestra.run.trial057.stdout:(27/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.8 MB/s | 66 kB 00:00 2026-02-12T09:06:54.541 INFO:teuthology.orchestra.run.trial057.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 178 kB/s | 25 kB 00:00 2026-02-12T09:06:54.602 INFO:teuthology.orchestra.run.trial057.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 810 kB/s | 49 kB 00:00 2026-02-12T09:06:54.643 INFO:teuthology.orchestra.run.trial057.stdout:(30/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.2 MB/s | 308 kB 00:00 2026-02-12T09:06:54.646 INFO:teuthology.orchestra.run.trial057.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 1.5 MB/s | 67 kB 00:00 2026-02-12T09:06:54.720 INFO:teuthology.orchestra.run.trial057.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 2.6 MB/s | 191 kB 00:00 2026-02-12T09:06:54.734 INFO:teuthology.orchestra.run.trial057.stdout:(33/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 9.0 MB/s | 838 kB 00:00 2026-02-12T09:06:54.757 INFO:teuthology.orchestra.run.trial057.stdout:(34/37): libarrow-9.0.0-15.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2026-02-12T09:06:54.842 INFO:teuthology.orchestra.run.trial057.stdout:(35/37): librados2-20.3.0-5161.g2401a109.el9.x8 30 MB/s | 3.3 MB 00:00 2026-02-12T09:06:54.845 INFO:teuthology.orchestra.run.trial057.stdout:(36/37): thrift-0.15.0-4.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2026-02-12T09:06:54.871 INFO:teuthology.orchestra.run.trial057.stdout:(37/37): librbd1-20.3.0-5161.g2401a109.el9.x86_ 24 MB/s | 2.7 MB 00:00 2026-02-12T09:06:54.874 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:06:54.875 INFO:teuthology.orchestra.run.trial057.stdout:Total 33 MB/s | 61 MB 00:01 2026-02-12T09:06:54.981 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:06:55.002 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:06:55.002 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:06:55.250 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:06:55.250 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:06:55.543 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:06:55.685 INFO:teuthology.orchestra.run.trial057.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-12T09:06:55.690 INFO:teuthology.orchestra.run.trial057.stdout: Upgrading : librados2-2:20.3.0-5161.g2401a109.el9.x86_64 2/39 2026-02-12T09:06:55.741 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librados2-2:20.3.0-5161.g2401a109.el9.x86_64 2/39 2026-02-12T09:06:55.742 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 3/39 2026-02-12T09:06:55.755 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 3/39 2026-02-12T09:06:55.758 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephfs-daemon-2:20.3.0-5161.g2401a109.el9.x86_6 4/39 2026-02-12T09:06:55.758 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephfs-proxy2-2:20.3.0-5161.g2401a109.el9.x86_6 5/39 2026-02-12T09:06:55.786 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5161.g2401a109.el9.x86_6 5/39 2026-02-12T09:06:55.788 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 6/39 2026-02-12T09:06:55.796 INFO:teuthology.orchestra.run.trial057.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-12T09:06:55.798 INFO:teuthology.orchestra.run.trial057.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-12T09:06:55.801 INFO:teuthology.orchestra.run.trial057.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-12T09:06:55.806 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-12T09:06:55.817 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-ceph-argparse-2:20.3.0-5161.g2401a109.el9. 11/39 2026-02-12T09:06:55.830 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 12/39 2026-02-12T09:06:55.831 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libradosstriper1-2:20.3.0-5161.g2401a109.el9.x86_6 13/39 2026-02-12T09:06:55.852 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5161.g2401a109.el9.x86_6 13/39 2026-02-12T09:06:55.856 INFO:teuthology.orchestra.run.trial057.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-12T09:06:55.887 INFO:teuthology.orchestra.run.trial057.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-12T09:06:55.891 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-12T09:06:55.897 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-12T09:06:55.899 INFO:teuthology.orchestra.run.trial057.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-12T09:06:55.903 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-12T09:06:56.019 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-12T09:06:56.021 INFO:teuthology.orchestra.run.trial057.stdout: Upgrading : librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 21/39 2026-02-12T09:06:56.047 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 21/39 2026-02-12T09:06:56.049 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 22/39 2026-02-12T09:06:56.055 INFO:teuthology.orchestra.run.trial057.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-12T09:06:56.311 INFO:teuthology.orchestra.run.trial057.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-12T09:06:56.312 INFO:teuthology.orchestra.run.trial057.stdout: Installing : librgw2-2:20.3.0-5161.g2401a109.el9.x86_64 25/39 2026-02-12T09:06:56.329 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librgw2-2:20.3.0-5161.g2401a109.el9.x86_64 25/39 2026-02-12T09:06:56.334 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 26/39 2026-02-12T09:06:56.339 INFO:teuthology.orchestra.run.trial057.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-12T09:06:56.341 INFO:teuthology.orchestra.run.trial057.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-12T09:06:56.342 INFO:teuthology.orchestra.run.trial057.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-12T09:06:56.361 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-12T09:06:56.476 INFO:teuthology.orchestra.run.trial057.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-12T09:06:56.479 INFO:teuthology.orchestra.run.trial057.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-12T09:06:56.751 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-12T09:06:56.751 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-12T09:06:56.751 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:06:56.796 INFO:teuthology.orchestra.run.trial057.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-12T09:06:56.798 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-ceph-common-2:20.3.0-5161.g2401a109.el9.x8 33/39 2026-02-12T09:06:57.756 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 34/39 2026-02-12T09:06:57.758 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 34/39 2026-02-12T09:06:58.030 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 34/39 2026-02-12T09:06:58.033 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 35/39 2026-02-12T09:06:58.074 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 35/39 2026-02-12T09:06:58.075 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-12T09:06:58.075 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-12T09:06:58.075 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:06:58.078 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-selinux-2:20.3.0-5161.g2401a109.el9.x86_64 36/39 2026-02-12T09:07:02.615 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5161.g2401a109.el9.x86_64 36/39 2026-02-12T09:07:02.615 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /sys 2026-02-12T09:07:02.615 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /proc 2026-02-12T09:07:02.615 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /mnt 2026-02-12T09:07:02.615 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /var/tmp 2026-02-12T09:07:02.615 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /home 2026-02-12T09:07:02.616 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /root 2026-02-12T09:07:02.616 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /tmp 2026-02-12T09:07:02.616 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:03.120 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 37/39 2026-02-12T09:07:03.144 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 37/39 2026-02-12T09:07:03.144 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:07:03.144 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-12T09:07:03.144 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T09:07:03.144 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T09:07:03.144 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:03.146 INFO:teuthology.orchestra.run.trial057.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-12T09:07:03.157 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-12T09:07:03.157 INFO:teuthology.orchestra.run.trial057.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T09:07:03.213 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T09:07:03.213 INFO:teuthology.orchestra.run.trial057.stdout:Creating group 'qat' with GID 992. 2026-02-12T09:07:03.213 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 1/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 2/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 3/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-selinux-2:20.3.0-5161.g2401a109.el9.x86_64 4/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephfs-daemon-2:20.3.0-5161.g2401a109.el9.x86_6 5/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5161.g2401a109.el9.x86_6 6/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 7/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libradosstriper1-2:20.3.0-5161.g2401a109.el9.x86_6 8/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librgw2-2:20.3.0-5161.g2401a109.el9.x86_64 9/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5161.g2401a109.el9. 10/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-ceph-common-2:20.3.0-5161.g2401a109.el9.x8 11/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 12/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 13/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 14/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 15/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 17/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-12T09:07:03.495 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-12T09:07:03.496 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-12T09:07:03.496 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-12T09:07:03.496 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-12T09:07:03.496 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librados2-2:20.3.0-5161.g2401a109.el9.x86_64 36/39 2026-02-12T09:07:03.496 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-12T09:07:03.496 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 38/39 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout:Upgraded: 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: librados2-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: ceph-selinux-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-daemon-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-proxy2-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: libradosstriper1-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-12T09:07:03.559 INFO:teuthology.orchestra.run.trial057.stdout: librgw2-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: python3-ceph-argparse-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: python3-ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: re2-1:20211101-20.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:03.560 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:03.599 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-test 2026-02-12T09:07:03.902 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:12 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:03.986 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repository Size 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: ceph-test x86_64 2:20.3.0-5161.g2401a109.el9 ceph 126 M 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout:Installing dependencies: 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: libasan8 x86_64 15.1.1-2.5.el9 appstream 523 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: libcephsqlite x86_64 2:20.3.0-5161.g2401a109.el9 ceph 153 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-12T09:07:03.987 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:03.988 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:03.988 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:03.988 INFO:teuthology.orchestra.run.trial057.stdout:Install 12 Packages 2026-02-12T09:07:03.988 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:03.988 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 131 M 2026-02-12T09:07:03.988 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 976 M 2026-02-12T09:07:03.988 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:04.288 INFO:teuthology.orchestra.run.trial057.stdout:(1/12): libcephsqlite-20.3.0-5161.g2401a109.el9 1.5 MB/s | 153 kB 00:00 2026-02-12T09:07:04.833 INFO:teuthology.orchestra.run.trial057.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 42 kB/s | 23 kB 00:00 2026-02-12T09:07:05.213 INFO:teuthology.orchestra.run.trial057.stdout:(3/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.0 MB/s | 2.1 MB 00:01 2026-02-12T09:07:05.375 INFO:teuthology.orchestra.run.trial057.stdout:(4/12): libasan8-15.1.1-2.5.el9.x86_64.rpm 3.2 MB/s | 523 kB 00:00 2026-02-12T09:07:05.401 INFO:teuthology.orchestra.run.trial057.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 6.6 MB/s | 176 kB 00:00 2026-02-12T09:07:05.428 INFO:teuthology.orchestra.run.trial057.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 8.6 MB/s | 233 kB 00:00 2026-02-12T09:07:05.487 INFO:teuthology.orchestra.run.trial057.stdout:(7/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 144 kB/s | 94 kB 00:00 2026-02-12T09:07:05.493 INFO:teuthology.orchestra.run.trial057.stdout:(8/12): protobuf-3.14.0-17.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2026-02-12T09:07:05.517 INFO:teuthology.orchestra.run.trial057.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.6 MB/s | 64 kB 00:00 2026-02-12T09:07:05.604 INFO:teuthology.orchestra.run.trial057.stdout:(10/12): socat-1.7.4.1-8.el9.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2026-02-12T09:07:05.668 INFO:teuthology.orchestra.run.trial057.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 811 kB/s | 122 kB 00:00 2026-02-12T09:07:06.628 INFO:teuthology.orchestra.run.trial057.stdout:(12/12): ceph-test-20.3.0-5161.g2401a109.el9.x8 52 MB/s | 126 MB 00:02 2026-02-12T09:07:06.629 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:06.629 INFO:teuthology.orchestra.run.trial057.stdout:Total 49 MB/s | 131 MB 00:02 2026-02-12T09:07:06.708 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:06.720 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:06.720 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:07.145 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:07.145 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:07.599 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:07.607 INFO:teuthology.orchestra.run.trial057.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-12T09:07:07.616 INFO:teuthology.orchestra.run.trial057.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-12T09:07:07.621 INFO:teuthology.orchestra.run.trial057.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-12T09:07:07.625 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-12T09:07:07.628 INFO:teuthology.orchestra.run.trial057.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-12T09:07:07.632 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-12T09:07:07.633 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libasan8-15.1.1-2.5.el9.x86_64 7/12 2026-02-12T09:07:07.650 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libasan8-15.1.1-2.5.el9.x86_64 7/12 2026-02-12T09:07:07.652 INFO:teuthology.orchestra.run.trial057.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-12T09:07:07.662 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-12T09:07:07.670 INFO:teuthology.orchestra.run.trial057.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-12T09:07:07.671 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephsqlite-2:20.3.0-5161.g2401a109.el9.x86_64 11/12 2026-02-12T09:07:14.547 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5161.g2401a109.el9.x86_64 11/12 2026-02-12T09:07:14.566 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 12/12 2026-02-12T09:07:15.322 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 12/12 2026-02-12T09:07:15.322 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 1/12 2026-02-12T09:07:15.322 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephsqlite-2:20.3.0-5161.g2401a109.el9.x86_64 2/12 2026-02-12T09:07:15.322 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-12T09:07:15.323 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-12T09:07:15.323 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-12T09:07:15.323 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libasan8-15.1.1-2.5.el9.x86_64 6/12 2026-02-12T09:07:15.323 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-12T09:07:15.323 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-12T09:07:15.323 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-12T09:07:15.323 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-12T09:07:15.323 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-12T09:07:15.377 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: libasan8-15.1.1-2.5.el9.x86_64 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: libcephsqlite-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-12T09:07:15.378 INFO:teuthology.orchestra.run.trial057.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-12T09:07:15.379 INFO:teuthology.orchestra.run.trial057.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-12T09:07:15.379 INFO:teuthology.orchestra.run.trial057.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-12T09:07:15.379 INFO:teuthology.orchestra.run.trial057.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-12T09:07:15.379 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:15.379 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:15.418 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph 2026-02-12T09:07:15.712 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:24 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:15.799 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:15.802 INFO:teuthology.orchestra.run.trial057.stdout:======================================================================================= 2026-02-12T09:07:15.802 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repository Size 2026-02-12T09:07:15.802 INFO:teuthology.orchestra.run.trial057.stdout:======================================================================================= 2026-02-12T09:07:15.802 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:15.802 INFO:teuthology.orchestra.run.trial057.stdout: ceph x86_64 2:20.3.0-5161.g2401a109.el9 ceph 6.5 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout:Installing dependencies: 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 57 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mds x86_64 2:20.3.0-5161.g2401a109.el9 ceph 2.0 M 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr x86_64 2:20.3.0-5161.g2401a109.el9 ceph 937 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 297 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mon x86_64 2:20.3.0-5161.g2401a109.el9 ceph 4.8 M 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd x86_64 2:20.3.0-5161.g2401a109.el9 ceph 163 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-classic x86_64 2:20.3.0-5161.g2401a109.el9 ceph 15 M 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-crimson x86_64 2:20.3.0-5161.g2401a109.el9 ceph 87 M 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 17 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: cephadm noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 1.0 M 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-12T09:07:15.803 INFO:teuthology.orchestra.run.trial057.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-12T09:07:15.804 INFO:teuthology.orchestra.run.trial057.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-12T09:07:15.805 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout:Installing weak dependencies: 2026-02-12T09:07:15.806 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 183 k 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 15 M 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 7.4 M 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 22 k 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-rook noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 50 k 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: ceph-volume noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 299 k 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout:======================================================================================= 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout:Install 96 Packages 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 189 M 2026-02-12T09:07:15.807 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 1.4 G 2026-02-12T09:07:15.808 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:16.135 INFO:teuthology.orchestra.run.trial057.stdout:(1/96): ceph-20.3.0-5161.g2401a109.el9.x86_64.r 128 kB/s | 6.5 kB 00:00 2026-02-12T09:07:16.221 INFO:teuthology.orchestra.run.trial057.stdout:(2/96): ceph-mgr-20.3.0-5161.g2401a109.el9.x86_ 6.7 MB/s | 937 kB 00:00 2026-02-12T09:07:16.245 INFO:teuthology.orchestra.run.trial057.stdout:(3/96): ceph-mds-20.3.0-5161.g2401a109.el9.x86_ 13 MB/s | 2.0 MB 00:00 2026-02-12T09:07:16.249 INFO:teuthology.orchestra.run.trial057.stdout:(4/96): ceph-osd-20.3.0-5161.g2401a109.el9.x86_ 5.7 MB/s | 163 kB 00:00 2026-02-12T09:07:16.373 INFO:teuthology.orchestra.run.trial057.stdout:(5/96): ceph-mon-20.3.0-5161.g2401a109.el9.x86_ 20 MB/s | 4.8 MB 00:00 2026-02-12T09:07:16.409 INFO:teuthology.orchestra.run.trial057.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5161.g24 1.6 MB/s | 57 kB 00:00 2026-02-12T09:07:16.451 INFO:teuthology.orchestra.run.trial057.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5161.g2401a109. 4.2 MB/s | 183 kB 00:00 2026-02-12T09:07:16.858 INFO:teuthology.orchestra.run.trial057.stdout:(8/96): ceph-osd-classic-20.3.0-5161.g2401a109. 24 MB/s | 15 MB 00:00 2026-02-12T09:07:17.207 INFO:teuthology.orchestra.run.trial057.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-51 21 MB/s | 7.4 MB 00:00 2026-02-12T09:07:17.226 INFO:teuthology.orchestra.run.trial057.stdout:(10/96): ceph-mgr-dashboard-20.3.0-5161.g2401a1 19 MB/s | 15 MB 00:00 2026-02-12T09:07:17.227 INFO:teuthology.orchestra.run.trial057.stdout:(11/96): ceph-mgr-k8sevents-20.3.0-5161.g2401a1 1.1 MB/s | 22 kB 00:00 2026-02-12T09:07:17.239 INFO:teuthology.orchestra.run.trial057.stdout:(12/96): ceph-mgr-modules-core-20.3.0-5161.g240 22 MB/s | 297 kB 00:00 2026-02-12T09:07:17.242 INFO:teuthology.orchestra.run.trial057.stdout:(13/96): ceph-mgr-rook-20.3.0-5161.g2401a109.el 3.2 MB/s | 50 kB 00:00 2026-02-12T09:07:17.254 INFO:teuthology.orchestra.run.trial057.stdout:(14/96): ceph-prometheus-alerts-20.3.0-5161.g24 1.1 MB/s | 17 kB 00:00 2026-02-12T09:07:17.263 INFO:teuthology.orchestra.run.trial057.stdout:(15/96): ceph-volume-20.3.0-5161.g2401a109.el9. 14 MB/s | 299 kB 00:00 2026-02-12T09:07:17.294 INFO:teuthology.orchestra.run.trial057.stdout:(16/96): cephadm-20.3.0-5161.g2401a109.el9.noar 26 MB/s | 1.0 MB 00:00 2026-02-12T09:07:17.877 INFO:teuthology.orchestra.run.trial057.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 69 kB/s | 40 kB 00:00 2026-02-12T09:07:18.108 INFO:teuthology.orchestra.run.trial057.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 312 kB/s | 72 kB 00:00 2026-02-12T09:07:18.182 INFO:teuthology.orchestra.run.trial057.stdout:(19/96): cryptsetup-2.8.1-2.el9.x86_64.rpm 381 kB/s | 350 kB 00:00 2026-02-12T09:07:18.490 INFO:teuthology.orchestra.run.trial057.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 600 kB/s | 184 kB 00:00 2026-02-12T09:07:18.619 INFO:teuthology.orchestra.run.trial057.stdout:(21/96): ceph-osd-crimson-20.3.0-5161.g2401a109 37 MB/s | 87 MB 00:02 2026-02-12T09:07:18.926 INFO:teuthology.orchestra.run.trial057.stdout:(22/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 580 kB/s | 253 kB 00:00 2026-02-12T09:07:19.096 INFO:teuthology.orchestra.run.trial057.stdout:(23/96): libgfortran-11.5.0-14.el9.x86_64.rpm 805 kB/s | 794 kB 00:00 2026-02-12T09:07:19.158 INFO:teuthology.orchestra.run.trial057.stdout:(24/96): python3-ply-3.11-14.el9.noarch.rpm 460 kB/s | 106 kB 00:00 2026-02-12T09:07:19.326 INFO:teuthology.orchestra.run.trial057.stdout:(25/96): python3-pycparser-2.20-6.el9.noarch.rp 587 kB/s | 135 kB 00:00 2026-02-12T09:07:19.466 INFO:teuthology.orchestra.run.trial057.stdout:(26/96): python3-pyparsing-2.4.7-9.el9.noarch.r 489 kB/s | 150 kB 00:00 2026-02-12T09:07:19.557 INFO:teuthology.orchestra.run.trial057.stdout:(27/96): python3-requests-2.25.1-10.el9.noarch. 549 kB/s | 126 kB 00:00 2026-02-12T09:07:19.851 INFO:teuthology.orchestra.run.trial057.stdout:(28/96): python3-urllib3-1.26.5-7.el9.noarch.rp 567 kB/s | 218 kB 00:00 2026-02-12T09:07:19.862 INFO:teuthology.orchestra.run.trial057.stdout:(29/96): unzip-6.0-59.el9.x86_64.rpm 595 kB/s | 182 kB 00:00 2026-02-12T09:07:20.233 INFO:teuthology.orchestra.run.trial057.stdout:(30/96): zip-3.0-35.el9.x86_64.rpm 695 kB/s | 266 kB 00:00 2026-02-12T09:07:20.432 INFO:teuthology.orchestra.run.trial057.stdout:(31/96): flexiblas-3.0.4-8.el9.x86_64.rpm 57 kB/s | 33 kB 00:00 2026-02-12T09:07:20.467 INFO:teuthology.orchestra.run.trial057.stdout:(32/96): python3-cryptography-36.0.1-5.el9.x86_ 690 kB/s | 1.2 MB 00:01 2026-02-12T09:07:20.920 INFO:teuthology.orchestra.run.trial057.stdout:(33/96): flexiblas-openblas-openmp-3.0.4-8.el9. 37 kB/s | 18 kB 00:00 2026-02-12T09:07:21.233 INFO:teuthology.orchestra.run.trial057.stdout:(34/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 322 kB/s | 246 kB 00:00 2026-02-12T09:07:21.279 INFO:teuthology.orchestra.run.trial057.stdout:(35/96): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 2.9 MB/s | 3.0 MB 00:01 2026-02-12T09:07:21.659 INFO:teuthology.orchestra.run.trial057.stdout:(36/96): lua-5.4.4-4.el9.x86_64.rpm 255 kB/s | 188 kB 00:00 2026-02-12T09:07:21.810 INFO:teuthology.orchestra.run.trial057.stdout:(37/96): openblas-0.3.29-1.el9.x86_64.rpm 73 kB/s | 42 kB 00:00 2026-02-12T09:07:22.394 INFO:teuthology.orchestra.run.trial057.stdout:(38/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 4.7 MB/s | 5.3 MB 00:01 2026-02-12T09:07:22.570 INFO:teuthology.orchestra.run.trial057.stdout:(39/96): python3-devel-3.9.25-2.el9.x86_64.rpm 322 kB/s | 244 kB 00:00 2026-02-12T09:07:22.785 INFO:teuthology.orchestra.run.trial057.stdout:(40/96): python3-babel-2.9.1-2.el9.noarch.rpm 5.3 MB/s | 6.0 MB 00:01 2026-02-12T09:07:23.158 INFO:teuthology.orchestra.run.trial057.stdout:(41/96): python3-jmespath-1.0.1-1.el9.noarch.rp 81 kB/s | 48 kB 00:00 2026-02-12T09:07:23.165 INFO:teuthology.orchestra.run.trial057.stdout:(42/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 323 kB/s | 249 kB 00:00 2026-02-12T09:07:23.511 INFO:teuthology.orchestra.run.trial057.stdout:(43/96): python3-libstoragemgmt-1.10.1-1.el9.x8 244 kB/s | 177 kB 00:00 2026-02-12T09:07:23.728 INFO:teuthology.orchestra.run.trial057.stdout:(44/96): python3-markupsafe-1.1.1-12.el9.x86_64 62 kB/s | 35 kB 00:00 2026-02-12T09:07:24.111 INFO:teuthology.orchestra.run.trial057.stdout:(45/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 1.3 MB/s | 1.2 MB 00:00 2026-02-12T09:07:24.559 INFO:teuthology.orchestra.run.trial057.stdout:(46/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 532 kB/s | 442 kB 00:00 2026-02-12T09:07:24.641 INFO:teuthology.orchestra.run.trial057.stdout:(47/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 5.4 MB/s | 6.1 MB 00:01 2026-02-12T09:07:24.746 INFO:teuthology.orchestra.run.trial057.stdout:(48/96): python3-packaging-20.9-5.el9.noarch.rp 121 kB/s | 77 kB 00:00 2026-02-12T09:07:25.335 INFO:teuthology.orchestra.run.trial057.stdout:(49/96): python3-protobuf-3.14.0-17.el9.noarch. 345 kB/s | 267 kB 00:00 2026-02-12T09:07:25.354 INFO:teuthology.orchestra.run.trial057.stdout:(50/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 222 kB/s | 159 kB 00:00 2026-02-12T09:07:25.527 INFO:teuthology.orchestra.run.trial057.stdout:(51/96): python3-pyasn1-modules-0.4.8-6.el9.noa 357 kB/s | 279 kB 00:00 2026-02-12T09:07:25.933 INFO:teuthology.orchestra.run.trial057.stdout:(52/96): python3-requests-oauthlib-1.3.0-12.el9 90 kB/s | 54 kB 00:00 2026-02-12T09:07:26.103 INFO:teuthology.orchestra.run.trial057.stdout:(53/96): python3-toml-0.10.2-6.el9.noarch.rpm 72 kB/s | 42 kB 00:00 2026-02-12T09:07:26.668 INFO:teuthology.orchestra.run.trial057.stdout:(54/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 257 kB/s | 189 kB 00:00 2026-02-12T09:07:26.765 INFO:teuthology.orchestra.run.trial057.stdout:(55/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 14 MB/s | 19 MB 00:01 2026-02-12T09:07:26.765 INFO:teuthology.orchestra.run.trial057.stdout:(56/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 136 kB/s | 90 kB 00:00 2026-02-12T09:07:26.998 INFO:teuthology.orchestra.run.trial057.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.3 MB/s | 551 kB 00:00 2026-02-12T09:07:27.030 INFO:teuthology.orchestra.run.trial057.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 604 kB/s | 19 kB 00:00 2026-02-12T09:07:27.064 INFO:teuthology.orchestra.run.trial057.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 4.3 MB/s | 151 kB 00:00 2026-02-12T09:07:27.101 INFO:teuthology.orchestra.run.trial057.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 15 MB/s | 548 kB 00:00 2026-02-12T09:07:27.104 INFO:teuthology.orchestra.run.trial057.stdout:(61/96): lua-devel-5.4.4-4.el9.x86_64.rpm 51 kB/s | 22 kB 00:00 2026-02-12T09:07:27.134 INFO:teuthology.orchestra.run.trial057.stdout:(62/96): python3-autocommand-2.2.2-8.el9.noarch 901 kB/s | 29 kB 00:00 2026-02-12T09:07:27.167 INFO:teuthology.orchestra.run.trial057.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2026-02-12T09:07:27.199 INFO:teuthology.orchestra.run.trial057.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 978 kB/s | 32 kB 00:00 2026-02-12T09:07:27.230 INFO:teuthology.orchestra.run.trial057.stdout:(65/96): python3-backports-tarfile-1.2.0-1.el9. 478 kB/s | 60 kB 00:00 2026-02-12T09:07:27.233 INFO:teuthology.orchestra.run.trial057.stdout:(66/96): python3-certifi-2023.05.07-4.el9.noarc 426 kB/s | 14 kB 00:00 2026-02-12T09:07:27.268 INFO:teuthology.orchestra.run.trial057.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 9.8 MB/s | 358 kB 00:00 2026-02-12T09:07:27.295 INFO:teuthology.orchestra.run.trial057.stdout:(68/96): python3-cheroot-10.0.1-4.el9.noarch.rp 2.6 MB/s | 173 kB 00:00 2026-02-12T09:07:27.304 INFO:teuthology.orchestra.run.trial057.stdout:(69/96): python3-google-auth-2.45.0-1.el9.noarc 7.0 MB/s | 254 kB 00:00 2026-02-12T09:07:27.339 INFO:teuthology.orchestra.run.trial057.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 4.0 MB/s | 144 kB 00:00 2026-02-12T09:07:27.375 INFO:teuthology.orchestra.run.trial057.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 3.9 MB/s | 139 kB 00:00 2026-02-12T09:07:27.408 INFO:teuthology.orchestra.run.trial057.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 1.6 MB/s | 56 kB 00:00 2026-02-12T09:07:27.441 INFO:teuthology.orchestra.run.trial057.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 328 kB/s | 11 kB 00:00 2026-02-12T09:07:27.461 INFO:teuthology.orchestra.run.trial057.stdout:(74/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2026-02-12T09:07:27.474 INFO:teuthology.orchestra.run.trial057.stdout:(75/96): python3-jaraco-classes-3.2.1-5.el9.noa 535 kB/s | 18 kB 00:00 2026-02-12T09:07:27.494 INFO:teuthology.orchestra.run.trial057.stdout:(76/96): python3-jaraco-collections-3.0.0-8.el9 705 kB/s | 23 kB 00:00 2026-02-12T09:07:27.507 INFO:teuthology.orchestra.run.trial057.stdout:(77/96): python3-jaraco-context-6.0.1-3.el9.noa 596 kB/s | 20 kB 00:00 2026-02-12T09:07:27.526 INFO:teuthology.orchestra.run.trial057.stdout:(78/96): python3-jaraco-functools-3.5.0-2.el9.n 598 kB/s | 19 kB 00:00 2026-02-12T09:07:27.540 INFO:teuthology.orchestra.run.trial057.stdout:(79/96): python3-jaraco-text-4.0.0-2.el9.noarch 799 kB/s | 26 kB 00:00 2026-02-12T09:07:27.574 INFO:teuthology.orchestra.run.trial057.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 2.3 MB/s | 79 kB 00:00 2026-02-12T09:07:27.594 INFO:teuthology.orchestra.run.trial057.stdout:(81/96): python3-kubernetes-26.1.0-3.el9.noarch 15 MB/s | 1.0 MB 00:00 2026-02-12T09:07:27.608 INFO:teuthology.orchestra.run.trial057.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.5 MB/s | 86 kB 00:00 2026-02-12T09:07:27.627 INFO:teuthology.orchestra.run.trial057.stdout:(83/96): python3-natsort-7.1.1-5.el9.noarch.rpm 1.7 MB/s | 58 kB 00:00 2026-02-12T09:07:27.641 INFO:teuthology.orchestra.run.trial057.stdout:(84/96): python3-portend-3.1.0-2.el9.noarch.rpm 499 kB/s | 16 kB 00:00 2026-02-12T09:07:27.661 INFO:teuthology.orchestra.run.trial057.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.6 MB/s | 90 kB 00:00 2026-02-12T09:07:27.675 INFO:teuthology.orchestra.run.trial057.stdout:(86/96): python3-repoze-lru-0.7-16.el9.noarch.r 926 kB/s | 31 kB 00:00 2026-02-12T09:07:27.695 INFO:teuthology.orchestra.run.trial057.stdout:(87/96): python3-routes-2.5.1-5.el9.noarch.rpm 5.3 MB/s | 188 kB 00:00 2026-02-12T09:07:27.709 INFO:teuthology.orchestra.run.trial057.stdout:(88/96): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2026-02-12T09:07:27.730 INFO:teuthology.orchestra.run.trial057.stdout:(89/96): python3-saml-1.16.0-1.el9.noarch.rpm 3.6 MB/s | 125 kB 00:00 2026-02-12T09:07:27.742 INFO:teuthology.orchestra.run.trial057.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 1.1 MB/s | 36 kB 00:00 2026-02-12T09:07:27.762 INFO:teuthology.orchestra.run.trial057.stdout:(91/96): python3-typing-extensions-4.15.0-1.el9 2.6 MB/s | 86 kB 00:00 2026-02-12T09:07:27.776 INFO:teuthology.orchestra.run.trial057.stdout:(92/96): python3-websocket-client-1.2.3-2.el9.n 2.6 MB/s | 90 kB 00:00 2026-02-12T09:07:27.795 INFO:teuthology.orchestra.run.trial057.stdout:(93/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.4 MB/s | 48 kB 00:00 2026-02-12T09:07:27.809 INFO:teuthology.orchestra.run.trial057.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 670 kB/s | 22 kB 00:00 2026-02-12T09:07:27.828 INFO:teuthology.orchestra.run.trial057.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 618 kB/s | 20 kB 00:00 2026-02-12T09:07:28.158 INFO:teuthology.orchestra.run.trial057.stdout:(96/96): protobuf-compiler-3.14.0-17.el9.x86_64 619 kB/s | 862 kB 00:01 2026-02-12T09:07:28.163 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:28.163 INFO:teuthology.orchestra.run.trial057.stdout:Total 15 MB/s | 189 MB 00:12 2026-02-12T09:07:28.566 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:28.603 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:28.603 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:29.291 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:29.292 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:30.009 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:30.011 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-12T09:07:30.014 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-12T09:07:30.035 INFO:teuthology.orchestra.run.trial057.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-12T09:07:30.069 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-12T09:07:30.076 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-12T09:07:30.078 INFO:teuthology.orchestra.run.trial057.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-12T09:07:30.083 INFO:teuthology.orchestra.run.trial057.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 7/96 2026-02-12T09:07:30.105 INFO:teuthology.orchestra.run.trial057.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-12T09:07:30.118 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-12T09:07:30.121 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-12T09:07:30.127 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-12T09:07:30.129 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-12T09:07:30.140 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-12T09:07:30.146 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-12T09:07:30.170 INFO:teuthology.orchestra.run.trial057.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-12T09:07:30.173 INFO:teuthology.orchestra.run.trial057.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-12T09:07:30.223 INFO:teuthology.orchestra.run.trial057.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-12T09:07:30.236 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-12T09:07:30.239 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-12T09:07:30.243 INFO:teuthology.orchestra.run.trial057.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-12T09:07:30.248 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-12T09:07:30.253 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-12T09:07:30.256 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-12T09:07:30.270 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-12T09:07:30.275 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-12T09:07:30.280 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-12T09:07:30.291 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-12T09:07:30.300 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-12T09:07:30.306 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-12T09:07:30.326 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-12T09:07:30.337 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-12T09:07:30.353 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-12T09:07:30.357 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-12T09:07:30.365 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-12T09:07:30.405 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-12T09:07:30.728 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-12T09:07:30.738 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-12T09:07:30.742 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-12T09:07:30.748 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-12T09:07:30.749 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-12T09:07:30.777 INFO:teuthology.orchestra.run.trial057.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-12T09:07:30.821 INFO:teuthology.orchestra.run.trial057.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-12T09:07:30.830 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-12T09:07:30.835 INFO:teuthology.orchestra.run.trial057.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-12T09:07:30.839 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-12T09:07:30.845 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-12T09:07:30.848 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-12T09:07:30.855 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-12T09:07:30.859 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-12T09:07:30.886 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-12T09:07:30.896 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-12T09:07:30.902 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-12T09:07:30.909 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-12T09:07:30.944 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-12T09:07:31.167 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 55/96 2026-02-12T09:07:31.190 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-12T09:07:31.194 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-12T09:07:31.253 INFO:teuthology.orchestra.run.trial057.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-12T09:07:31.255 INFO:teuthology.orchestra.run.trial057.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-12T09:07:31.276 INFO:teuthology.orchestra.run.trial057.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 60/96 2026-02-12T09:07:31.601 INFO:teuthology.orchestra.run.trial057.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 61/96 2026-02-12T09:07:31.673 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-12T09:07:32.341 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-12T09:07:32.358 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-12T09:07:32.368 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-12T09:07:32.375 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-12T09:07:32.388 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-12T09:07:32.403 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-12T09:07:32.478 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-12T09:07:32.487 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-12T09:07:32.510 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-12T09:07:32.540 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-12T09:07:32.590 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-12T09:07:32.596 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-12T09:07:32.600 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-12T09:07:32.601 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-12T09:07:32.619 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T09:07:32.619 INFO:teuthology.orchestra.run.trial057.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-12T09:07:32.619 INFO:teuthology.orchestra.run.trial057.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-12T09:07:32.619 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:32.628 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T09:07:32.654 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T09:07:32.654 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-12T09:07:32.654 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:32.690 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-12T09:07:33.290 INFO:teuthology.orchestra.run.trial057.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 79/96 2026-02-12T09:07:38.728 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_6 80/96 2026-02-12T09:07:38.792 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_6 81/96 2026-02-12T09:07:38.794 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch 82/96 2026-02-12T09:07:38.808 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch 82/96 2026-02-12T09:07:38.809 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:07:38.809 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-12T09:07:38.809 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:38.819 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 83/96 2026-02-12T09:07:38.842 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 83/96 2026-02-12T09:07:38.842 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:07:38.843 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-12T09:07:38.843 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T09:07:38.843 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T09:07:38.843 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:38.902 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: cephadm-2:20.3.0-5161.g2401a109.el9.noarch 84/96 2026-02-12T09:07:38.904 INFO:teuthology.orchestra.run.trial057.stdout: Installing : cephadm-2:20.3.0-5161.g2401a109.el9.noarch 84/96 2026-02-12T09:07:38.913 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5161.g2401a109.el9 85/96 2026-02-12T09:07:38.937 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5161.g2401a109.el 86/96 2026-02-12T09:07:38.938 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarc 87/96 2026-02-12T09:07:40.526 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarc 87/96 2026-02-12T09:07:40.533 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noa 88/96 2026-02-12T09:07:40.995 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noa 88/96 2026-02-12T09:07:40.996 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a 89/96 2026-02-12T09:07:41.009 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a 89/96 2026-02-12T09:07:41.010 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5161.g2401a109.el9.noa 90/96 2026-02-12T09:07:41.072 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5161.g2401a109.el9.noa 90/96 2026-02-12T09:07:41.113 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5161.g2401a109.el9. 91/96 2026-02-12T09:07:41.115 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 92/96 2026-02-12T09:07:41.136 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 92/96 2026-02-12T09:07:41.137 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:07:41.137 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-12T09:07:41.137 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T09:07:41.137 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T09:07:41.137 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:41.148 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch 93/96 2026-02-12T09:07:41.370 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch 93/96 2026-02-12T09:07:41.371 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mon-2:20.3.0-5161.g2401a109.el9.x86_64 94/96 2026-02-12T09:07:41.392 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mon-2:20.3.0-5161.g2401a109.el9.x86_64 94/96 2026-02-12T09:07:41.392 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:07:41.392 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-12T09:07:41.392 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T09:07:41.392 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T09:07:41.392 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:41.477 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mds-2:20.3.0-5161.g2401a109.el9.x86_64 95/96 2026-02-12T09:07:41.497 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mds-2:20.3.0-5161.g2401a109.el9.x86_64 95/96 2026-02-12T09:07:41.497 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:07:41.497 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-12T09:07:41.497 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T09:07:41.497 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T09:07:41.497 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:41.498 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-2:20.3.0-5161.g2401a109.el9.x86_64 96/96 2026-02-12T09:07:41.505 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_6 96/96 2026-02-12T09:07:41.528 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_6 96/96 2026-02-12T09:07:42.886 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-2:20.3.0-5161.g2401a109.el9.x86_64 96/96 2026-02-12T09:07:42.886 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-2:20.3.0-5161.g2401a109.el9.x86_64 1/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mds-2:20.3.0-5161.g2401a109.el9.x86_64 2/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 3/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mon-2:20.3.0-5161.g2401a109.el9.x86_64 4/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 5/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_6 6/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_6 7/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5161.g2401a109.el 8/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarc 9/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noa 10/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a 11/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5161.g2401a109.el9.noa 12/96 2026-02-12T09:07:42.887 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5161.g2401a109.el9. 13/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch 14/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5161.g2401a109.el9 15/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch 16/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : cephadm-2:20.3.0-5161.g2401a109.el9.noarch 17/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 18/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-12T09:07:42.888 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 32/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 33/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 34/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-12T09:07:42.889 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 40/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-12T09:07:42.890 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-12T09:07:42.891 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-12T09:07:42.892 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-12T09:07:42.893 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-12T09:07:42.894 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-12T09:07:42.894 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-12T09:07:42.894 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-12T09:07:42.894 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-12T09:07:42.894 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-12T09:07:42.952 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-12T09:07:42.952 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:42.952 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:42.952 INFO:teuthology.orchestra.run.trial057.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-12T09:07:42.952 INFO:teuthology.orchestra.run.trial057.stdout: ceph-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:42.952 INFO:teuthology.orchestra.run.trial057.stdout: ceph-grafana-dashboards-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.952 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mds-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:42.952 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-k8sevents-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-modules-core-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mon-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-prometheus-alerts-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.953 INFO:teuthology.orchestra.run.trial057.stdout: ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: cephadm-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: lua-5.4.4-4.el9.x86_64 2026-02-12T09:07:42.954 INFO:teuthology.orchestra.run.trial057.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-12T09:07:42.955 INFO:teuthology.orchestra.run.trial057.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-12T09:07:42.956 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-12T09:07:42.957 INFO:teuthology.orchestra.run.trial057.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-ply-3.11-14.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-12T09:07:42.958 INFO:teuthology.orchestra.run.trial057.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: unzip-6.0-59.el9.x86_64 2026-02-12T09:07:42.959 INFO:teuthology.orchestra.run.trial057.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-12T09:07:42.960 INFO:teuthology.orchestra.run.trial057.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-12T09:07:42.960 INFO:teuthology.orchestra.run.trial057.stdout: zip-3.0-35.el9.x86_64 2026-02-12T09:07:42.960 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:42.960 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:43.013 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-base 2026-02-12T09:07:43.312 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:52 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:43.377 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:43.395 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:43.396 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:43.396 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:43.416 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install cephadm 2026-02-12T09:07:43.718 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:52 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:43.785 INFO:teuthology.orchestra.run.trial057.stdout:Package cephadm-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:07:43.803 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:43.803 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:43.804 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:43.824 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-immutable-object-cache 2026-02-12T09:07:44.125 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:53 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repo Size 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5161.g2401a109.el9 ceph 149 k 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout:Install 1 Package 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:44.209 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 149 k 2026-02-12T09:07:44.210 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 455 k 2026-02-12T09:07:44.210 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:44.317 INFO:teuthology.orchestra.run.trial057.stdout:ceph-immutable-object-cache-20.3.0-5161.g2401a1 1.4 MB/s | 149 kB 00:00 2026-02-12T09:07:44.317 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:44.318 INFO:teuthology.orchestra.run.trial057.stdout:Total 1.3 MB/s | 149 kB 00:00 2026-02-12T09:07:44.318 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:44.323 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:44.323 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:44.356 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:44.356 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:44.411 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:44.413 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5161.g2401a109. 1/1 2026-02-12T09:07:44.436 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5161.g2401a109. 1/1 2026-02-12T09:07:44.436 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:07:44.436 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-12T09:07:44.436 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:44.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5161.g2401a109. 1/1 2026-02-12T09:07:44.761 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:44.761 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:44.761 INFO:teuthology.orchestra.run.trial057.stdout: ceph-immutable-object-cache-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:44.761 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:44.762 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:44.790 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr 2026-02-12T09:07:45.094 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:54 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:45.160 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:45.179 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:45.179 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:45.179 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:45.200 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr-dashboard 2026-02-12T09:07:45.494 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:54 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:45.559 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:07:45.579 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:45.579 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:45.579 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:45.599 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-12T09:07:45.903 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:54 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:45.970 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:07:45.989 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:45.989 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:45.989 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:46.010 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr-rook 2026-02-12T09:07:46.309 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:55 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:46.375 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:07:46.394 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:46.394 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:46.394 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:46.415 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr-cephadm 2026-02-12T09:07:46.713 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:55 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:46.780 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:07:46.799 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:46.799 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:46.799 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:46.820 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-osd 2026-02-12T09:07:47.119 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:56 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:47.184 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:47.203 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:47.203 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:47.203 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:47.224 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-osd-classic 2026-02-12T09:07:47.524 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:56 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:47.588 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:47.607 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:47.607 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:47.607 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:47.627 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-fuse 2026-02-12T09:07:47.933 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:56 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repo Size 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout: ceph-fuse x86_64 2:20.3.0-5161.g2401a109.el9 ceph 914 k 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:Install 1 Package 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 914 k 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 2.9 M 2026-02-12T09:07:48.018 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:48.737 INFO:teuthology.orchestra.run.trial057.stdout:ceph-fuse-20.3.0-5161.g2401a109.el9.x86_64.rpm 1.2 MB/s | 914 kB 00:00 2026-02-12T09:07:48.737 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:48.737 INFO:teuthology.orchestra.run.trial057.stdout:Total 1.2 MB/s | 914 kB 00:00 2026-02-12T09:07:48.738 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:48.743 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:48.743 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:48.778 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:48.778 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:48.869 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:48.886 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:49.119 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:49.176 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:49.176 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:49.176 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:49.176 INFO:teuthology.orchestra.run.trial057.stdout: ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:49.176 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:49.177 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:49.205 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-volume 2026-02-12T09:07:49.511 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:58 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:49.576 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:07:49.594 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:49.594 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:49.594 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:49.615 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install librados-devel 2026-02-12T09:07:49.913 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:58 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:49.996 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:49.996 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repo Size 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout: librados-devel x86_64 2:20.3.0-5161.g2401a109.el9 ceph 124 k 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout:Install 1 Package 2026-02-12T09:07:49.997 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:49.998 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 124 k 2026-02-12T09:07:49.998 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 434 k 2026-02-12T09:07:49.998 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:50.346 INFO:teuthology.orchestra.run.trial057.stdout:librados-devel-20.3.0-5161.g2401a109.el9.x86_64 355 kB/s | 124 kB 00:00 2026-02-12T09:07:50.346 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:50.347 INFO:teuthology.orchestra.run.trial057.stdout:Total 353 kB/s | 124 kB 00:00 2026-02-12T09:07:50.347 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:50.349 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:50.349 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:50.382 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:50.383 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:50.437 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:50.454 INFO:teuthology.orchestra.run.trial057.stdout: Installing : librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:50.538 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:50.598 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:50.598 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:50.598 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:50.599 INFO:teuthology.orchestra.run.trial057.stdout: librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:50.599 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:50.599 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:50.628 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install libcephfs2 2026-02-12T09:07:50.928 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:59 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:50.993 INFO:teuthology.orchestra.run.trial057.stdout:Package libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:51.010 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:51.011 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:51.011 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:51.032 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install libcephfs-devel 2026-02-12T09:07:51.334 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:00 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:51.417 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:51.417 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:51.417 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repo Size 2026-02-12T09:07:51.417 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:51.417 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:51.417 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-devel x86_64 2:20.3.0-5161.g2401a109.el9 ceph 37 k 2026-02-12T09:07:51.417 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:51.418 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:51.418 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:51.418 INFO:teuthology.orchestra.run.trial057.stdout:Install 1 Package 2026-02-12T09:07:51.418 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:51.418 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 37 k 2026-02-12T09:07:51.418 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 171 k 2026-02-12T09:07:51.418 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:51.691 INFO:teuthology.orchestra.run.trial057.stdout:libcephfs-devel-20.3.0-5161.g2401a109.el9.x86_6 136 kB/s | 37 kB 00:00 2026-02-12T09:07:51.691 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:51.691 INFO:teuthology.orchestra.run.trial057.stdout:Total 136 kB/s | 37 kB 00:00 2026-02-12T09:07:51.691 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:51.694 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:51.694 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:51.697 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:51.697 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:51.719 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:51.735 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:51.797 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:51.854 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:51.854 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:51.854 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:51.854 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:51.854 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:51.854 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:51.880 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install librados2 2026-02-12T09:07:52.179 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:01 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:52.246 INFO:teuthology.orchestra.run.trial057.stdout:Package librados2-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:52.263 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:52.264 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:52.264 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:52.285 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install librbd1 2026-02-12T09:07:52.586 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:01 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:52.653 INFO:teuthology.orchestra.run.trial057.stdout:Package librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:52.670 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:52.670 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:52.670 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:52.692 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install python3-rados 2026-02-12T09:07:52.987 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:01 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:53.054 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:53.071 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:53.072 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:53.072 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:53.094 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install python3-rgw 2026-02-12T09:07:53.397 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:02 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:53.461 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:53.478 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:53.479 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:53.479 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:53.500 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install python3-cephfs 2026-02-12T09:07:53.800 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:02 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:53.865 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:53.882 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:53.883 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:53.883 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:53.904 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install python3-rbd 2026-02-12T09:07:54.202 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:03 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:54.269 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:54.286 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:54.287 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:54.287 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:54.308 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install rbd-fuse 2026-02-12T09:07:54.615 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:03 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout: Package Architecture Version Repository Size 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout: rbd-fuse x86_64 2:20.3.0-5161.g2401a109.el9 ceph 87 k 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:Install 1 Package 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 87 k 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 236 k 2026-02-12T09:07:54.700 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:55.026 INFO:teuthology.orchestra.run.trial057.stdout:rbd-fuse-20.3.0-5161.g2401a109.el9.x86_64.rpm 266 kB/s | 87 kB 00:00 2026-02-12T09:07:55.026 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:55.027 INFO:teuthology.orchestra.run.trial057.stdout:Total 265 kB/s | 87 kB 00:00 2026-02-12T09:07:55.027 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:55.031 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:55.031 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:55.063 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:55.064 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:55.115 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:55.131 INFO:teuthology.orchestra.run.trial057.stdout: Installing : rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:55.212 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:55.270 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:55.270 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:55.270 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:55.270 INFO:teuthology.orchestra.run.trial057.stdout: rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:55.270 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:55.270 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:55.298 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install rbd-mirror 2026-02-12T09:07:55.601 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:04 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repo Size 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout: rbd-mirror x86_64 2:20.3.0-5161.g2401a109.el9 ceph 2.6 M 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout:Install 1 Package 2026-02-12T09:07:55.683 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:55.684 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 2.6 M 2026-02-12T09:07:55.684 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 11 M 2026-02-12T09:07:55.684 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:57.318 INFO:teuthology.orchestra.run.trial057.stdout:rbd-mirror-20.3.0-5161.g2401a109.el9.x86_64.rpm 1.6 MB/s | 2.6 MB 00:01 2026-02-12T09:07:57.319 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:57.319 INFO:teuthology.orchestra.run.trial057.stdout:Total 1.6 MB/s | 2.6 MB 00:01 2026-02-12T09:07:57.319 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:57.324 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:57.324 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:57.363 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:57.363 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:57.528 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:57.529 INFO:teuthology.orchestra.run.trial057.stdout: Installing : rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:57.552 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:57.552 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:07:57.552 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-12T09:07:57.552 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T09:07:57.552 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T09:07:57.552 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:57.853 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:57.853 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:57.853 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:57.853 INFO:teuthology.orchestra.run.trial057.stdout: rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:57.853 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:57.853 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:57.883 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install rbd-nbd 2026-02-12T09:07:58.189 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:07 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:58.270 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout: Package Architecture Version Repository Size 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout: rbd-nbd x86_64 2:20.3.0-5161.g2401a109.el9 ceph 166 k 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:Install 1 Package 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 166 k 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 496 k 2026-02-12T09:07:58.271 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:07:58.649 INFO:teuthology.orchestra.run.trial057.stdout:rbd-nbd-20.3.0-5161.g2401a109.el9.x86_64.rpm 441 kB/s | 166 kB 00:00 2026-02-12T09:07:58.649 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:07:58.649 INFO:teuthology.orchestra.run.trial057.stdout:Total 440 kB/s | 166 kB 00:00 2026-02-12T09:07:58.649 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:07:58.654 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:07:58.654 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:07:58.686 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:07:58.687 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:07:58.747 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:07:58.764 INFO:teuthology.orchestra.run.trial057.stdout: Installing : rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:58.845 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:58.902 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:07:58.903 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:58.903 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:07:58.903 INFO:teuthology.orchestra.run.trial057.stdout: rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:07:58.903 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:58.903 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:58.931 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-osd-crimson 2026-02-12T09:07:59.237 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:08 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:59.302 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:07:59.320 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:59.321 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:07:59.321 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:07:59.342 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-12T09:07:59.646 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:01:08 ago on Thu 12 Feb 2026 09:06:51 AM UTC. 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repo Size 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5161.g2401a109.el9 ceph 429 M 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:Installing weak dependencies: 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout: ceph-debugsource x86_64 2:20.3.0-5161.g2401a109.el9 ceph 12 M 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:Install 2 Packages 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 441 M 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 2.7 G 2026-02-12T09:07:59.729 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-12T09:08:03.605 INFO:teuthology.orchestra.run.trial057.stdout:(1/2): ceph-debugsource-20.3.0-5161.g2401a109.e 3.1 MB/s | 12 MB 00:03 2026-02-12T09:08:26.710 INFO:teuthology.orchestra.run.trial057.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5161.g 16 MB/s | 429 MB 00:26 2026-02-12T09:08:26.712 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:08:26.712 INFO:teuthology.orchestra.run.trial057.stdout:Total 16 MB/s | 441 MB 00:26 2026-02-12T09:08:26.712 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-12T09:08:26.717 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-12T09:08:26.717 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-12T09:08:28.077 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-12T09:08:28.078 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-12T09:08:31.647 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-12T09:08:53.673 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-debugsource-2:20.3.0-5161.g2401a109.el9.x86_64 1/2 2026-02-12T09:08:53.691 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5161.g2401a109.e 2/2 2026-02-12T09:08:55.666 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5161.g2401a109.e 2/2 2026-02-12T09:08:55.666 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-debugsource-2:20.3.0-5161.g2401a109.el9.x86_64 1/2 2026-02-12T09:08:55.723 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5161.g2401a109.e 2/2 2026-02-12T09:08:55.723 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:08:55.724 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-12T09:08:55.724 INFO:teuthology.orchestra.run.trial057.stdout: ceph-debugsource-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:08:55.724 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:08:55.724 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:08:55.724 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:08:55.798 DEBUG:teuthology.parallel:result is None 2026-02-12T09:08:55.799 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:08:55.966 DEBUG:teuthology.orchestra.run.trial057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T09:08:55.988 INFO:teuthology.orchestra.run.trial057.stdout:20.3.0-5161.g2401a109.el9 2026-02-12T09:08:55.989 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5161.g2401a109.el9 2026-02-12T09:08:55.989 INFO:teuthology.task.install:The correct ceph version 20.3.0-5161.g2401a109 is installed. 2026-02-12T09:08:55.991 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-12T09:08:55.991 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:08:55.991 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-12T09:08:56.054 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-12T09:08:56.054 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:08:56.054 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/usr/bin/daemon-helper 2026-02-12T09:08:56.117 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-12T09:08:56.179 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-12T09:08:56.179 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:08:56.179 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-12T09:08:56.242 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-12T09:08:56.306 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-12T09:08:56.307 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:08:56.307 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/usr/bin/stdin-killer 2026-02-12T09:08:56.371 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-12T09:08:56.436 INFO:teuthology.run_tasks:Running task pexec... 2026-02-12T09:08:56.454 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-12T09:08:56.455 DEBUG:teuthology.orchestra.run.trial057:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-12T09:08:56.478 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:08:56.478 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-12T09:08:56.529 DEBUG:teuthology.parallel:result is None 2026-02-12T09:08:56.530 INFO:teuthology.run_tasks:Running task install... 2026-02-12T09:08:56.543 DEBUG:teuthology.task.install:project ceph 2026-02-12T09:08:56.544 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}} 2026-02-12T09:08:56.544 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'} 2026-02-12T09:08:56.544 INFO:teuthology.task.install:Using flavor: debug 2026-02-12T09:08:56.553 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T09:08:56.553 INFO:teuthology.task.install:extra packages: [] 2026-02-12T09:08:56.553 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875', 'tag': None, 'wait_for_package': False} 2026-02-12T09:08:56.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:08:56.724 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/ 2026-02-12T09:08:56.724 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5161.g2401a109 2026-02-12T09:08:56.999 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T09:08:56.999 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:08:56.999 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T09:08:57.024 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-12T09:08:57.024 DEBUG:teuthology.orchestra.run.trial057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2401a109e2ac85dfa20fc57328da663aa754c875/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T09:08:57.089 DEBUG:teuthology.orchestra.run.trial057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T09:08:57.154 DEBUG:teuthology.orchestra.run.trial057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T09:08:57.210 INFO:teuthology.orchestra.run.trial057.stdout:check_obsoletes = 1 2026-02-12T09:08:57.220 DEBUG:teuthology.orchestra.run.trial057:> sudo yum clean all 2026-02-12T09:08:57.379 INFO:teuthology.orchestra.run.trial057.stdout:56 files removed 2026-02-12T09:08:57.394 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-radosgw 2026-02-12T09:08:57.798 INFO:teuthology.orchestra.run.trial057.stdout:ceph packages for x86_64 374 kB/s | 94 kB 00:00 2026-02-12T09:08:58.010 INFO:teuthology.orchestra.run.trial057.stdout:ceph noarch packages 130 kB/s | 24 kB 00:00 2026-02-12T09:08:58.205 INFO:teuthology.orchestra.run.trial057.stdout:ceph source packages 12 kB/s | 2.1 kB 00:00 2026-02-12T09:08:59.560 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.9 MB 00:01 2026-02-12T09:09:25.444 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 27 MB 00:25 2026-02-12T09:09:28.819 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 7.8 MB 00:01 2026-02-12T09:09:30.607 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 20 kB 00:01 2026-02-12T09:09:31.684 INFO:teuthology.orchestra.run.trial057.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:01 2026-02-12T09:09:34.882 INFO:teuthology.orchestra.run.trial057.stdout:lab-extras 2.3 MB/s | 50 kB 00:00 2026-02-12T09:09:36.136 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:36.154 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:36.155 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:36.155 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:36.178 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-test 2026-02-12T09:09:36.478 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:02 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:36.543 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:36.560 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:36.561 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:36.561 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:36.583 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph 2026-02-12T09:09:36.879 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:02 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:36.943 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:36.960 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:36.961 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:36.961 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:36.982 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-base 2026-02-12T09:09:37.284 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:03 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:37.350 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:37.367 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:37.368 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:37.368 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:37.389 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install cephadm 2026-02-12T09:09:37.693 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:03 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:37.758 INFO:teuthology.orchestra.run.trial057.stdout:Package cephadm-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:09:37.775 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:37.776 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:37.776 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:37.798 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-immutable-object-cache 2026-02-12T09:09:38.100 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:38.166 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-immutable-object-cache-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:38.183 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:38.183 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:38.184 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:38.206 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr 2026-02-12T09:09:38.509 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:38.574 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:38.591 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:38.592 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:38.592 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:38.614 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr-dashboard 2026-02-12T09:09:38.911 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:38.976 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:09:38.993 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:38.994 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:38.994 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:39.016 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-12T09:09:39.317 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:05 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:39.384 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:09:39.401 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:39.402 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:39.402 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:39.424 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr-rook 2026-02-12T09:09:39.724 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:05 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:39.790 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:09:39.807 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:39.808 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:39.808 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:39.829 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-mgr-cephadm 2026-02-12T09:09:40.127 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:06 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:40.192 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:09:40.209 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:40.210 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:40.210 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:40.232 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-osd 2026-02-12T09:09:40.531 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:06 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:40.596 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:40.614 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:40.614 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:40.615 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:40.636 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-osd-classic 2026-02-12T09:09:40.935 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:06 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:41.002 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:41.019 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:41.020 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:41.020 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:41.041 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-fuse 2026-02-12T09:09:41.342 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:07 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:41.408 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:41.425 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:41.426 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:41.426 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:41.447 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-volume 2026-02-12T09:09:41.745 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:07 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:41.809 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:09:41.826 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:41.826 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:41.826 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:41.848 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install librados-devel 2026-02-12T09:09:42.144 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:08 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:42.210 INFO:teuthology.orchestra.run.trial057.stdout:Package librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:42.228 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:42.228 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:42.228 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:42.251 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install libcephfs2 2026-02-12T09:09:42.547 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:08 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:42.612 INFO:teuthology.orchestra.run.trial057.stdout:Package libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:42.630 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:42.630 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:42.630 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:42.652 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install libcephfs-devel 2026-02-12T09:09:42.957 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:08 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:43.023 INFO:teuthology.orchestra.run.trial057.stdout:Package libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:43.040 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:43.041 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:43.041 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:43.064 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install librados2 2026-02-12T09:09:43.364 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:09 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:43.428 INFO:teuthology.orchestra.run.trial057.stdout:Package librados2-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:43.446 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:43.446 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:43.447 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:43.469 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install librbd1 2026-02-12T09:09:43.777 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:09 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:43.841 INFO:teuthology.orchestra.run.trial057.stdout:Package librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:43.859 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:43.859 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:43.859 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:43.882 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install python3-rados 2026-02-12T09:09:44.182 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:10 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:44.263 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:44.264 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:44.264 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:44.286 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install python3-rgw 2026-02-12T09:09:44.585 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:10 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:44.649 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:44.666 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:44.666 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:44.666 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:44.688 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install python3-cephfs 2026-02-12T09:09:44.990 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:10 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:45.055 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:45.072 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:45.072 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:45.073 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:45.095 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install python3-rbd 2026-02-12T09:09:45.397 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:11 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:45.461 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:45.478 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:45.479 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:45.479 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:45.501 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install rbd-fuse 2026-02-12T09:09:45.805 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:11 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:45.869 INFO:teuthology.orchestra.run.trial057.stdout:Package rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:45.887 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:45.887 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:45.888 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:45.909 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install rbd-mirror 2026-02-12T09:09:46.216 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:12 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:46.280 INFO:teuthology.orchestra.run.trial057.stdout:Package rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:46.298 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:46.298 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:46.298 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:46.321 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install rbd-nbd 2026-02-12T09:09:46.623 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:12 ago on Thu 12 Feb 2026 09:09:34 AM UTC. 2026-02-12T09:09:46.689 INFO:teuthology.orchestra.run.trial057.stdout:Package rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:09:46.706 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-12T09:09:46.706 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-12T09:09:46.707 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-12T09:09:46.731 DEBUG:teuthology.parallel:result is None 2026-02-12T09:09:46.732 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:09:46.902 DEBUG:teuthology.orchestra.run.trial057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T09:09:46.922 INFO:teuthology.orchestra.run.trial057.stdout:20.3.0-5161.g2401a109.el9 2026-02-12T09:09:46.922 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5161.g2401a109.el9 2026-02-12T09:09:46.922 INFO:teuthology.task.install:The correct ceph version 20.3.0-5161.g2401a109 is installed. 2026-02-12T09:09:46.924 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-12T09:09:46.925 INFO:teuthology.run_tasks:Running task ceph... 2026-02-12T09:09:47.023 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-12T09:09:47.023 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod 777 /var/log/ceph 2026-02-12T09:09:47.048 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-12T09:09:47.048 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-12T09:09:47.111 INFO:tasks.ceph:Creating extra log directories... 2026-02-12T09:09:47.111 DEBUG:teuthology.orchestra.run.trial057:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-12T09:09:47.183 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-12T09:09:47.183 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-12T09:09:47.183 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41601', 'branch': 'main', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/radosbench_omap_write}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '41601', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c'}}, 'owner': 'scheduled_matan@soko04.front.sepia.ceph.com', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 9353, 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c', 'targets': {'trial057.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHXdTLia5nq7hSbgAURcBVolnepOvqJYXd+BQz8BQ6nuVui+R6GjDga6XdvVAe3u4su4oGQCyCGLNLA2gZytTuw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'radosbench': {'clients': ['client.0'], 'objectsize': 4096, 'size': 4096, 'time': 300, 'write-omap': True}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'c433f1062990a0488dc29a553589bc609a460691', 'timestamp': '2026-02-09_15:50:17', 'tube': 'trial', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.2161924'} 2026-02-12T09:09:47.184 DEBUG:teuthology.orchestra.run.trial057:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-12T09:09:47.232 DEBUG:teuthology.orchestra.run.trial057:> sudo install -d -m0777 -- /var/run/ceph 2026-02-12T09:09:47.296 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:09:47.296 DEBUG:teuthology.orchestra.run.trial057:> dd if=/scratch_devs of=/dev/stdout 2026-02-12T09:09:47.352 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-12T09:09:47.353 DEBUG:teuthology.orchestra.run.trial057:> stat /dev/vg_nvme/lv_1 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout:Access: 2026-02-12 09:07:57.692230981 +0000 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout:Modify: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout:Change: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.410 INFO:teuthology.orchestra.run.trial057.stdout: Birth: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.411 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-12T09:09:47.472 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records in 2026-02-12T09:09:47.473 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records out 2026-02-12T09:09:47.473 INFO:teuthology.orchestra.run.trial057.stderr:512 bytes copied, 0.000143807 s, 3.6 MB/s 2026-02-12T09:09:47.474 DEBUG:teuthology.orchestra.run.trial057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-12T09:09:47.531 DEBUG:teuthology.orchestra.run.trial057:> stat /dev/vg_nvme/lv_2 2026-02-12T09:09:47.587 INFO:teuthology.orchestra.run.trial057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-12T09:09:47.587 INFO:teuthology.orchestra.run.trial057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T09:09:47.588 INFO:teuthology.orchestra.run.trial057.stdout:Device: 6h/6d Inode: 1305 Links: 1 2026-02-12T09:09:47.588 INFO:teuthology.orchestra.run.trial057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T09:09:47.588 INFO:teuthology.orchestra.run.trial057.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T09:09:47.588 INFO:teuthology.orchestra.run.trial057.stdout:Access: 2026-02-12 09:07:57.692230981 +0000 2026-02-12T09:09:47.588 INFO:teuthology.orchestra.run.trial057.stdout:Modify: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.588 INFO:teuthology.orchestra.run.trial057.stdout:Change: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.588 INFO:teuthology.orchestra.run.trial057.stdout: Birth: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.589 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-12T09:09:47.649 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records in 2026-02-12T09:09:47.649 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records out 2026-02-12T09:09:47.649 INFO:teuthology.orchestra.run.trial057.stderr:512 bytes copied, 0.000143055 s, 3.6 MB/s 2026-02-12T09:09:47.651 DEBUG:teuthology.orchestra.run.trial057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-12T09:09:47.708 DEBUG:teuthology.orchestra.run.trial057:> stat /dev/vg_nvme/lv_3 2026-02-12T09:09:47.764 INFO:teuthology.orchestra.run.trial057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-12T09:09:47.764 INFO:teuthology.orchestra.run.trial057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T09:09:47.764 INFO:teuthology.orchestra.run.trial057.stdout:Device: 6h/6d Inode: 1306 Links: 1 2026-02-12T09:09:47.764 INFO:teuthology.orchestra.run.trial057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T09:09:47.764 INFO:teuthology.orchestra.run.trial057.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T09:09:47.765 INFO:teuthology.orchestra.run.trial057.stdout:Access: 2026-02-12 09:07:57.692230981 +0000 2026-02-12T09:09:47.765 INFO:teuthology.orchestra.run.trial057.stdout:Modify: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.765 INFO:teuthology.orchestra.run.trial057.stdout:Change: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.765 INFO:teuthology.orchestra.run.trial057.stdout: Birth: 2026-02-12 09:06:56.779306826 +0000 2026-02-12T09:09:47.765 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-12T09:09:47.825 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records in 2026-02-12T09:09:47.826 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records out 2026-02-12T09:09:47.826 INFO:teuthology.orchestra.run.trial057.stderr:512 bytes copied, 0.00013455 s, 3.8 MB/s 2026-02-12T09:09:47.826 DEBUG:teuthology.orchestra.run.trial057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-12T09:09:47.885 DEBUG:teuthology.orchestra.run.trial057:> stat /dev/vg_nvme/lv_4 2026-02-12T09:09:47.941 INFO:teuthology.orchestra.run.trial057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-12T09:09:47.941 INFO:teuthology.orchestra.run.trial057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T09:09:47.941 INFO:teuthology.orchestra.run.trial057.stdout:Device: 6h/6d Inode: 1290 Links: 1 2026-02-12T09:09:47.942 INFO:teuthology.orchestra.run.trial057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T09:09:47.942 INFO:teuthology.orchestra.run.trial057.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T09:09:47.942 INFO:teuthology.orchestra.run.trial057.stdout:Access: 2026-02-12 09:07:57.693230959 +0000 2026-02-12T09:09:47.942 INFO:teuthology.orchestra.run.trial057.stdout:Modify: 2026-02-12 09:06:56.778306843 +0000 2026-02-12T09:09:47.942 INFO:teuthology.orchestra.run.trial057.stdout:Change: 2026-02-12 09:06:56.778306843 +0000 2026-02-12T09:09:47.942 INFO:teuthology.orchestra.run.trial057.stdout: Birth: 2026-02-12 09:06:56.778306843 +0000 2026-02-12T09:09:47.942 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-12T09:09:48.005 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records in 2026-02-12T09:09:48.005 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records out 2026-02-12T09:09:48.005 INFO:teuthology.orchestra.run.trial057.stderr:512 bytes copied, 0.000130202 s, 3.9 MB/s 2026-02-12T09:09:48.006 DEBUG:teuthology.orchestra.run.trial057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-12T09:09:48.064 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T09:09:48.065 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-12T09:09:48.065 INFO:tasks.ceph:Generating config... 2026-02-12T09:09:48.067 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-12T09:09:48.067 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-12T09:09:48.067 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-12T09:09:48.067 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-12T09:09:48.067 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-12T09:09:48.068 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-12T09:09:48.069 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-12T09:09:48.069 INFO:tasks.ceph:[global] auth client required = none 2026-02-12T09:09:48.069 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-12T09:09:48.069 INFO:tasks.ceph:[global] auth service required = none 2026-02-12T09:09:48.069 INFO:tasks.ceph:[global] auth supported = none 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-12T09:09:48.070 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-12T09:09:48.071 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-12T09:09:48.072 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-12T09:09:48.073 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-12T09:09:48.074 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-12T09:09:48.075 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-12T09:09:48.075 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-12T09:09:48.075 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-12T09:09:48.075 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-12T09:09:48.075 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-12T09:09:48.075 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-12T09:09:48.075 INFO:tasks.ceph:Setting up mon.a... 2026-02-12T09:09:48.075 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-12T09:09:48.144 INFO:teuthology.orchestra.run.trial057.stdout:creating /etc/ceph/ceph.keyring 2026-02-12T09:09:48.147 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-12T09:09:48.232 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-12T09:09:48.296 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.57')] 2026-02-12T09:09:48.297 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.57', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-12T09:09:48.299 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:09:48.299 DEBUG:teuthology.orchestra.run.trial057:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-12T09:09:48.352 DEBUG:teuthology.orchestra.run.trial057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.57 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:09:48.433 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:09:48.433+0000 7f9eb3e32f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:09:48.433 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:09:48.433+0000 7f9eb3e32f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:09:48.433 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:09:48.433+0000 7f9eb3e32f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:monmaptool: generated fsid 2d25ad73-62b5-4644-9a52-c68c542c153c 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:setting min_mon_release = tentacle 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:epoch 0 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:fsid 2d25ad73-62b5-4644-9a52-c68c542c153c 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:last_changed 2026-02-12T09:09:48.435128+0000 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:created 2026-02-12T09:09:48.435128+0000 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:min_mon_release 20 (tentacle) 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:election_strategy: 1 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:0: [v2:10.20.193.57:3300/0,v1:10.20.193.57:6789/0] mon.a 2026-02-12T09:09:48.434 INFO:teuthology.orchestra.run.trial057.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-12T09:09:48.435 DEBUG:teuthology.orchestra.run.trial057:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-12T09:09:48.492 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2d25ad73-62b5-4644-9a52-c68c542c153c... 2026-02-12T09:09:48.493 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout:[global] 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: chdir = "" 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: auth supported = none 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: filestore xattr use omap = true 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: mon clock drift allowed = 1.000 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: osd crush chooseleaf type = 0 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: auth debug = true 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: ms die on old message = true 2026-02-12T09:09:48.572 INFO:teuthology.orchestra.run.trial057.stdout: ms die on bug = true 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon pg warn max object skew = 0 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: # disable pg_autoscaler by default for new pools 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: osd pool default size = 2 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon osd allow primary affinity = true 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon osd allow pg remap = true 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on legacy crush tunables = false 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on crush straw calc version zero = false 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on no sortbitwise = false 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on osd down out interval zero = false 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on too few osds = false 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon_allow_pool_size_one = true 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: osd default data pool replay window = 5 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon allow pool delete = true 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon cluster log file level = debug 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: debug asserts on shutdown = true 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon health detail to clog = false 2026-02-12T09:09:48.573 INFO:teuthology.orchestra.run.trial057.stdout: mon host = 10.20.193.57 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: auth client required = none 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: auth cluster required = none 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: auth service required = none 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug asok = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug auth = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug bdev = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug bluefs = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug bluestore = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug buffer = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug civetweb = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug client = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug compressor = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug context = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug crush = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug crypto = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug dpdk = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug eventtrace = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug filer = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug filestore = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug finisher = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug fuse = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug heartbeatmap = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug javaclient = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug journal = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug journaler = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug kstore = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug leveldb = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug lockdep = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug mds = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug mds balancer = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug mds locker = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug mds log = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug mds log expire = 0/0 2026-02-12T09:09:48.574 INFO:teuthology.orchestra.run.trial057.stdout: debug mds migrator = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug memdb = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug mgr = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug mgrc = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug mon = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug monc = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug ms = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug objclass = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug objectcacher = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug objecter = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug optracker = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug osd = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug paxos = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug perfcounter = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rados = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rbd = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rbd mirror = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rbd replay = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug refs = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rgw = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rgw lifecycle = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rgw notification = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rgw sync = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug rocksdb = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug striper = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug throttle = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug timer = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: debug tp = 0/0 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: ms client mode = crc 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: ms cluster mode = crc 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: ms mon client mode = crc 2026-02-12T09:09:48.575 INFO:teuthology.orchestra.run.trial057.stdout: ms mon cluster mode = crc 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: ms mon service mode = crc 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: ms service mode = crc 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: fsid = 2d25ad73-62b5-4644-9a52-c68c542c153c 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout:[osd] 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd journal size = 100 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd scrub load threshold = 5.0 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd scrub max interval = 600 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd mclock profile = high_recovery_ops 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd mclock skip benchmark = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd recover clone overlap = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd recovery max chunk = 1048576 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd debug shutdown = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd debug op order = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd debug verify stray on activate = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd debug trim objects = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd open classes on start = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd debug pg log writeout = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd deep scrub update digest min age = 30 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: osd map max advance = 10 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: journal zero on create = true 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: filestore ondisk finisher threads = 3 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: filestore apply finisher threads = 3 2026-02-12T09:09:48.576 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: bdev debug aio = true 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: osd debug misdirected ops = true 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug monc = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: crimson cpu num = 4 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: crimson osd obc lru size = 10 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug filestore = 0/0 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug journal = 0/0 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug ms = 0/0 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug osd = 0/0 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore backref = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore cache = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore cleaner = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore device = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore epm = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore fixedkv tree = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore journal = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore lba = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore odata = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore omap = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore onode = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore t = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore tm = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: osd objectstore = seastore 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: osd shutdown pgref assert = True 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: seastore cachepin size pershard = 64M 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: seastore cachepin type = 2Q 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: seastore max concurrent transactions = 8 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout:[mgr] 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug ms = 1 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug mgr = 20 2026-02-12T09:09:48.577 INFO:teuthology.orchestra.run.trial057.stdout: debug mon = 20 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: debug auth = 20 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon reweight min pgs per osd = 4 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon reweight min bytes per osd = 10 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mgr/telemetry/nag = false 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout:[mon] 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: debug ms = 0/0 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: debug mon = 0/0 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: debug paxos = 0/0 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: debug auth = 20 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon data avail warn = 5 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon mgr mkfs grace = 240 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon reweight min pgs per osd = 4 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon osd reporter subtree level = osd 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon osd prime pg temp = true 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon reweight min bytes per osd = 10 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: auth mon ticket ttl = 660 # 11m 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: auth service ticket ttl = 240 # 4m 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: # don't complain about insecure global_id in the test suite 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: # 1m isn't quite enough 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon_down_mkfs_grace = 2m 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_filestore_osds = false 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: osd pool default crimson = True 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:09:48.578 INFO:teuthology.orchestra.run.trial057.stdout:[client] 2026-02-12T09:09:48.579 INFO:teuthology.orchestra.run.trial057.stdout: rgw cache enabled = true 2026-02-12T09:09:48.579 INFO:teuthology.orchestra.run.trial057.stdout: rgw enable ops log = true 2026-02-12T09:09:48.579 INFO:teuthology.orchestra.run.trial057.stdout: rgw enable usage log = true 2026-02-12T09:09:48.579 INFO:teuthology.orchestra.run.trial057.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-12T09:09:48.579 INFO:teuthology.orchestra.run.trial057.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-12T09:09:48.579 INFO:teuthology.orchestra.run.trial057.stdout:[mon.a] 2026-02-12T09:09:48.582 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-12T09:09:48.582 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-12T09:09:48.673 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-12T09:09:48.673 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:09:48.673 DEBUG:teuthology.orchestra.run.trial057:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-12T09:09:48.690 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:09:48.690 DEBUG:teuthology.orchestra.run.trial057:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-12T09:09:48.747 INFO:tasks.ceph:Sending monmap to node ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:09:48.747 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:09:48.747 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-12T09:09:48.747 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-12T09:09:48.821 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:09:48.822 DEBUG:teuthology.orchestra.run.trial057:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:09:48.878 INFO:tasks.ceph:Setting up mon nodes... 2026-02-12T09:09:48.878 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-12T09:09:48.878 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-12T09:09:48.967 INFO:teuthology.orchestra.run.trial057.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-12T09:09:48.970 INFO:tasks.ceph:Setting up mds nodes... 2026-02-12T09:09:48.971 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-12T09:09:48.971 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-12T09:09:49.013 INFO:teuthology.orchestra.run.trial057.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-12T09:09:49.024 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-12T09:09:49.025 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-12T09:09:49.025 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-12T09:09:49.088 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T09:09:49.089 INFO:tasks.ceph:role: osd.0 2026-02-12T09:09:49.089 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:09:49.089 DEBUG:teuthology.orchestra.run.trial057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-12T09:09:49.153 INFO:teuthology.orchestra.run.trial057.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T09:09:49.153 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T09:09:49.153 INFO:teuthology.orchestra.run.trial057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T09:09:49.153 INFO:teuthology.orchestra.run.trial057.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T09:09:49.153 INFO:teuthology.orchestra.run.trial057.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T09:09:49.153 INFO:teuthology.orchestra.run.trial057.stdout: = sunit=0 swidth=0 blks 2026-02-12T09:09:49.153 INFO:teuthology.orchestra.run.trial057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T09:09:49.154 INFO:teuthology.orchestra.run.trial057.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T09:09:49.154 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T09:09:49.154 INFO:teuthology.orchestra.run.trial057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T09:09:49.724 INFO:teuthology.orchestra.run.trial057.stdout:Discarding blocks...Done. 2026-02-12T09:09:49.726 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial057.front.sepia.ceph.com -o noatime 2026-02-12T09:09:49.726 DEBUG:teuthology.orchestra.run.trial057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-12T09:09:49.794 DEBUG:teuthology.orchestra.run.trial057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-12T09:09:49.861 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-12T09:09:49.925 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T09:09:49.925 INFO:tasks.ceph:role: osd.1 2026-02-12T09:09:49.925 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:09:49.926 DEBUG:teuthology.orchestra.run.trial057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-12T09:09:49.989 INFO:teuthology.orchestra.run.trial057.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T09:09:49.989 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T09:09:49.990 INFO:teuthology.orchestra.run.trial057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T09:09:49.990 INFO:teuthology.orchestra.run.trial057.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T09:09:49.990 INFO:teuthology.orchestra.run.trial057.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T09:09:49.990 INFO:teuthology.orchestra.run.trial057.stdout: = sunit=0 swidth=0 blks 2026-02-12T09:09:49.990 INFO:teuthology.orchestra.run.trial057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T09:09:49.990 INFO:teuthology.orchestra.run.trial057.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T09:09:49.990 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T09:09:49.990 INFO:teuthology.orchestra.run.trial057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T09:09:50.546 INFO:teuthology.orchestra.run.trial057.stdout:Discarding blocks...Done. 2026-02-12T09:09:50.548 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial057.front.sepia.ceph.com -o noatime 2026-02-12T09:09:50.549 DEBUG:teuthology.orchestra.run.trial057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-12T09:09:50.616 DEBUG:teuthology.orchestra.run.trial057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-12T09:09:50.684 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-12T09:09:50.748 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T09:09:50.748 INFO:tasks.ceph:role: osd.2 2026-02-12T09:09:50.749 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:09:50.749 DEBUG:teuthology.orchestra.run.trial057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-12T09:09:50.814 INFO:teuthology.orchestra.run.trial057.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T09:09:50.814 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T09:09:50.814 INFO:teuthology.orchestra.run.trial057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T09:09:50.814 INFO:teuthology.orchestra.run.trial057.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T09:09:50.814 INFO:teuthology.orchestra.run.trial057.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T09:09:50.814 INFO:teuthology.orchestra.run.trial057.stdout: = sunit=0 swidth=0 blks 2026-02-12T09:09:50.815 INFO:teuthology.orchestra.run.trial057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T09:09:50.815 INFO:teuthology.orchestra.run.trial057.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T09:09:50.815 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T09:09:50.815 INFO:teuthology.orchestra.run.trial057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T09:09:51.371 INFO:teuthology.orchestra.run.trial057.stdout:Discarding blocks...Done. 2026-02-12T09:09:51.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial057.front.sepia.ceph.com -o noatime 2026-02-12T09:09:51.373 DEBUG:teuthology.orchestra.run.trial057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-12T09:09:51.441 DEBUG:teuthology.orchestra.run.trial057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-12T09:09:51.508 DEBUG:teuthology.orchestra.run.trial057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:09:51.656 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T09:09:51.713 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:51,715 seastar - Reactor backend: linux-aio 2026-02-12T09:09:51.725 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:51,726 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:09:51.725 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:51,726 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:09:51.727 INFO:teuthology.orchestra.run.trial057.stderr:==21233==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:09:51.787 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-12 09:09:51,788 [shard 0:main] seastar - reactor::drain 2026-02-12T09:09:53.865 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:09:53.865 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-12T09:09:53.866 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-12T09:09:53.866 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-12T09:09:53.866 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:09:53.866 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-12T09:09:53.885 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T09:09:53.885 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T09:09:53.929 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T09:09:53.929 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T09:09:53.936 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:53,937 seastar - Reactor backend: linux-aio 2026-02-12T09:09:53.950 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:53,952 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:09:53.950 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:53,952 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:09:53.954 INFO:teuthology.orchestra.run.trial057.stderr:==21231==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:09:53.954 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:53,956 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T09:09:54.096 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0xcb01c3 0x1124f62 0xf2dc91 0x9c3d9d0 0x9c3dccd 0x9c3df40 0x9c3e1f6 0x9c3eca5 0x9c3f1a3 0x9c3f7d3 0x9c3f93d 0x9d1fcad 0x9d32ab0 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.097 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1584d 0x10904af 0xf2dc9c 0x9c3d803 0x9c3dccd 0x9c3df40 0x9c3fc67 0x9d1ee32 0x9d1f5ab 0x9d1fa96 0x9d1fe29 0x9d32ab0 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.097 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x9c4fed1 0x9c4ff8d 0x9d0fd7b 0x9d0ff10 0x9d102a2 0x9d11889 0x9d12ba4 0x9d12fc4 0x9d2f84f 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.129 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x795f0e7 0x79b691b 0x79b6ac4 0x78237c2 0x7823941 0x78a0883 0x6f918a5 0x754759e 0x75476f2 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f674ac 0x9be50b1 0x9be58ec 0xeced0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46b04 2026-02-12T09:09:54.194 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ed3 /lib64/libasan.so.8+0xc6611 /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f674ac 0x9be50b1 0x9be58ec 0xeced0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46b04 2026-02-12T09:09:54.234 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15215 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.234 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a513ca 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.235 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a5022e 0x7d70884 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.363 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151ec 0x9bf9ddd 0x9d26260 0x9d2b950 0x9d2bf8e 0x9d3645b 0x788188f 0x7bd6cd5 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.363 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x9c88f28 0x9cf4c96 0x9d36476 0x788188f 0x7bd72da 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.364 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x316a758 0x9c3df2e 0x9c3e1f6 0x9c3eca5 0x9c3f1a3 0x9c3f7d3 0x9c3f93d 0x9d1fcad 0x9d32ab0 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.446 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151d9 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f674ac 0x9be50b1 0x9be58ec 0xeced0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46b04 2026-02-12T09:09:54.460 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a50bd5 0x7d708a6 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.460 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a510b7 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:54.461 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x29fc44c 0x2a51240 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:56.673 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:09:56.674 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-12T09:09:56.674 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-12T09:09:56.674 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-12T09:09:56.674 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:09:56.674 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-12T09:09:56.710 DEBUG:teuthology.orchestra.run.trial057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-12T09:09:56.736 DEBUG:teuthology.orchestra.run.trial057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:09:56.889 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T09:09:56.940 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:56,941 seastar - Reactor backend: linux-aio 2026-02-12T09:09:56.947 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:56,949 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:09:56.948 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:56,949 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:09:56.950 INFO:teuthology.orchestra.run.trial057.stderr:==21296==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:09:57.009 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-12 09:09:57,011 [shard 0:main] seastar - reactor::drain 2026-02-12T09:09:59.103 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:09:59.103 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-12T09:09:59.103 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-12T09:09:59.104 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-12T09:09:59.104 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:09:59.104 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-12T09:09:59.123 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T09:09:59.123 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T09:09:59.166 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T09:09:59.166 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T09:09:59.173 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:59,174 seastar - Reactor backend: linux-aio 2026-02-12T09:09:59.182 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:59,183 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:09:59.182 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:59,183 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:09:59.185 INFO:teuthology.orchestra.run.trial057.stderr:==21294==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:09:59.186 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:09:59,187 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T09:09:59.333 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb1472 0xb69317 0x1090438 0xf2dc9c 0x9c3d803 0x9c3dccd 0x9c3df40 0x9c3e1f6 0x9c3e7b8 0x9c46c5f 0x9c4733f 0x9d2fe8c 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.333 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x101faa /lib64/libasan.so.8+0xc3ed /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc65 0xe0b375 0xe0bd08 0x9d12fdc 0x9d2f84f 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.333 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x9c3d935 0x9c3dccd 0x9c3df40 0x9c3fc67 0x9c401b4 0x9c403b3 0x9d256d5 0x9d25e30 0x9d300be 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.365 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x1124ef1 0xf2dc91 0x9c3d9d0 0x9c3dccd 0x9c3df40 0x9c3fc67 0x9c401b4 0x9c403b3 0x9d256d5 0x9d25e30 0x9d300be 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bdb876 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f914c6 0x754759e 0x75476f2 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f674ac 0x9be50b1 0x9be58ec 0xeced0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46b04 2026-02-12T09:09:59.433 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x12976b /lib64/libasan.so.8+0xc6631 /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f674ac 0x9be50b1 0x9be58ec 0xeced0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46b04 2026-02-12T09:09:59.470 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a5116c 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.471 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x29fc44a 0x2a51240 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.471 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65f2 /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.599 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x158af 0x9c370c9 0x9d08328 0x9d347a4 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.600 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x109034b 0xf2dc9c 0x9c3d9d0 0x9c3dccd 0x9c3df40 0x9c3e1f6 0x9c3eca5 0x9c3f1a3 0x9c3f7d3 0x9c3f93d 0x9d1fcad 0x9d32ab0 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.600 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb0938 /lib64/libasan.so.8+0xb1477 0xb69317 0x1090438 0xf2dc9c 0x9c4222a 0x9c424d7 0x9d2f9c9 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.682 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a5116c 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f674ac 0x9be50b1 0x9be58ec 0xeced0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46b04 2026-02-12T09:09:59.696 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x3623 /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.697 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc663b /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:09:59.697 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x7d7086f 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:01.909 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:10:01.909 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-12T09:10:01.909 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-12T09:10:01.909 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-12T09:10:01.909 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:10:01.909 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-12T09:10:01.946 DEBUG:teuthology.orchestra.run.trial057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-12T09:10:01.970 DEBUG:teuthology.orchestra.run.trial057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:10:02.123 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T09:10:02.174 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:10:02,175 seastar - Reactor backend: linux-aio 2026-02-12T09:10:02.186 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:10:02,187 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:10:02.186 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:10:02,187 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:10:02.188 INFO:teuthology.orchestra.run.trial057.stderr:==21360==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:10:02.249 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-12 09:10:02,250 [shard 0:main] seastar - reactor::drain 2026-02-12T09:10:04.287 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:10:04.288 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-12T09:10:04.288 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-12T09:10:04.288 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-12T09:10:04.288 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:10:04.288 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-12T09:10:04.307 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T09:10:04.308 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T09:10:04.346 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T09:10:04.346 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T09:10:04.353 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:10:04,354 seastar - Reactor backend: linux-aio 2026-02-12T09:10:04.364 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:10:04,365 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:10:04.364 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:10:04,365 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:10:04.368 INFO:teuthology.orchestra.run.trial057.stderr:==21358==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:10:04.368 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-12 09:10:04,369 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T09:10:04.510 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x7149ded 0x78818cc 0x7bd6cd5 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.510 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x9c4e489 0x9d2b9bc 0x9d2bf8e 0x9d3645b 0x788188f 0x7bd72da 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.511 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7fc 0x9b1228b 0x9b8fa26 0x9b8fb3c 0x9b8fc18 0x9b8fe94 0x9b9043b 0x9b9083f 0x9b90afc 0x9b91944 0x9d08598 0x9d08691 0x7bd7317 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.511 INFO:teuthology.orchestra.run.trial057.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9e3f4a65 0xffffffff9e3f555e 0xffffffff9e3fcc78 0xffffffff9e424923 0xffffffff9e424ae5 0xffffffff9e3c9858 0xffffffff9e3d3b7b 0xffffffff9e3d3fdb 0xffffffff9e0910d2 0xffffffff9ed4d451 0xffffffff9ee00ba2 2026-02-12T09:10:04.608 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc6631 /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f674ac 0x9be50b1 0x9be58ec 0xeced0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46b04 2026-02-12T09:10:04.647 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a5108b 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.648 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a513cd 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.648 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x29fc44a 0x2a51240 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.773 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0xcbb398 0x9bdceca 0x9d12eb3 0x9d2f84f 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.773 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119128 /lib64/libasan.so.8+0xc5f4 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc65 0xe0b375 0xe0b5ab 0x7bd71fa 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.773 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7fc 0x9b1228b 0x9b8fa26 0x9b8fb3c 0x9b8fc18 0x9b902d4 0x9b8ff4d 0x9b9043b 0x9b9083f 0x9b90afc 0x9d32705 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.773 INFO:teuthology.orchestra.run.trial057.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9e7280f7 0xffffffff9e3f7fa6 0xffffffff9e3fb960 0xffffffff9e3fccc1 0xffffffff9e424923 0xffffffff9e424ae5 0xffffffff9e3c9858 0xffffffff9e3d3b7b 0xffffffff9e3d3fdb 0xffffffff9e0910d2 0xffffffff9ed4d451 0xffffffff9ee00ba2 2026-02-12T09:10:04.855 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65f2 /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f674ac 0x9be50b1 0x9be58ec 0xeced0d /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb46b04 2026-02-12T09:10:04.869 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ed3 /lib64/libasan.so.8+0xc6611 /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.870 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6622 /lib64/libasan.so.8+0x151b7 0x2a5129f 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:04.870 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x2a51146 0x7d7086b 0x7d8ce10 0x7e2ea95 0x7dae38f 0x7864ca5 0x786599a 0x799acdc 0x7823dae 0x789ee50 0x78e75e3 0x78e785c 0x7988a34 0x7988b93 0x799ae3f 0x7855926 0x79776a3 0x7977732 0x79777d6 0x73d42f8 0x73f4757 0x73f93f8 0x753a894 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:10:07.004 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:10:07.005 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-12T09:10:07.005 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-12T09:10:07.005 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-12T09:10:07.005 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-12T09:10:07.005 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-12T09:10:07.039 DEBUG:teuthology.orchestra.run.trial057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-12T09:10:07.065 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-12T09:10:07.066 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:07.066 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-12T09:10:07.132 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:07.132 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-12T09:10:07.196 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:07.196 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-12T09:10:07.258 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:07.258 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-12T09:10:07.322 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:07.322 DEBUG:teuthology.orchestra.run.trial057:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-12T09:10:07.379 INFO:tasks.ceph:Adding keys to all mons... 2026-02-12T09:10:07.380 DEBUG:teuthology.orchestra.run.trial057:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-12T09:10:07.441 INFO:teuthology.orchestra.run.trial057.stdout:[mgr.x] 2026-02-12T09:10:07.441 INFO:teuthology.orchestra.run.trial057.stdout: key = AQDcmI1pm+TCORAAgXDKIdyli3hn5XcDv6pTjA== 2026-02-12T09:10:07.441 INFO:teuthology.orchestra.run.trial057.stdout:[osd.0] 2026-02-12T09:10:07.442 INFO:teuthology.orchestra.run.trial057.stdout: key = AQDimI1pnARYARAAuiVnA9iZHzRPyqySWp+TXA== 2026-02-12T09:10:07.442 INFO:teuthology.orchestra.run.trial057.stdout:[osd.1] 2026-02-12T09:10:07.442 INFO:teuthology.orchestra.run.trial057.stdout: key = AQDnmI1pxPpPDxAAIeOIRKFQU41E5borOOVgMQ== 2026-02-12T09:10:07.442 INFO:teuthology.orchestra.run.trial057.stdout:[osd.2] 2026-02-12T09:10:07.442 INFO:teuthology.orchestra.run.trial057.stdout: key = AQDsmI1p9635GRAA6y8Bn76p33jqRF+/ekPkgg== 2026-02-12T09:10:07.442 INFO:teuthology.orchestra.run.trial057.stdout:[client.0] 2026-02-12T09:10:07.442 INFO:teuthology.orchestra.run.trial057.stdout: key = AQDdmI1pLAjlABAAqIaLIH1CUoCIBfuELayXwA== 2026-02-12T09:10:07.442 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-12T09:10:07.531 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T09:10:07.580 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T09:10:07.628 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T09:10:07.675 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-12T09:10:07.723 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-12T09:10:07.724 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-12T09:10:07.748 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-12T09:10:07.835 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:07.835+0000 7f1099cfdc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:07.835 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:07.835+0000 7f1099cfdc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:07.842 DEBUG:teuthology.orchestra.run.trial057:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-12T09:10:07.865 DEBUG:teuthology.orchestra.run.trial057:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:10:07.921 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-12T09:10:07.921 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-12T09:10:07.921 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-12T09:10:07.963 INFO:tasks.ceph.mon.a:Started 2026-02-12T09:10:07.963 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-12T09:10:07.963 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-12T09:10:07.963 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-12T09:10:07.964 INFO:tasks.ceph.mgr.x:Started 2026-02-12T09:10:07.964 DEBUG:tasks.ceph:set 0 configs 2026-02-12T09:10:07.964 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph config dump 2026-02-12T09:10:08.028 INFO:tasks.ceph.mgr.x.trial057.stderr:2026-02-12T09:10:08.028+0000 7f25741c0fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.029 INFO:tasks.ceph.mon.a.trial057.stderr:2026-02-12T09:10:08.029+0000 7f8c6cb23c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.029 INFO:tasks.ceph.mon.a.trial057.stderr:2026-02-12T09:10:08.029+0000 7f8c6cb23c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.029 INFO:tasks.ceph.mon.a.trial057.stderr:2026-02-12T09:10:08.029+0000 7f8c6cb23c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.041 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:08.039+0000 7ff44d0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.043 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:08.043+0000 7ff44d0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.075 INFO:teuthology.orchestra.run.trial057.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-12T09:10:08.082 INFO:tasks.ceph:Setting crush tunables to default 2026-02-12T09:10:08.083 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd crush tunables default 2026-02-12T09:10:08.157 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:08.154+0000 7f6d31c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.158 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:08.158+0000 7f6d31c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.190 INFO:teuthology.orchestra.run.trial057.stderr:adjusted tunables profile to default 2026-02-12T09:10:08.198 INFO:tasks.ceph:check_enable_crimson: True 2026-02-12T09:10:08.198 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-12T09:10:08.198 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-12T09:10:08.231 INFO:tasks.ceph.mgr.x.trial057.stderr:2026-02-12T09:10:08.231+0000 7f25741c0fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.232 INFO:tasks.ceph.mgr.x.trial057.stderr:2026-02-12T09:10:08.232+0000 7f25741c0fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.272 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:08.270+0000 7f63a041a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.274 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:08.274+0000 7f63a041a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:08.313 INFO:tasks.ceph.mon.a.trial057.stderr:2026-02-12T09:10:08.313+0000 7f8c64df1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-12T09:10:08.314 INFO:tasks.ceph.mon.a.trial057.stderr:Please be aware that this feature is experimental, untested, 2026-02-12T09:10:08.314 INFO:tasks.ceph.mon.a.trial057.stderr:unsupported, and may result in data corruption, data loss, 2026-02-12T09:10:08.314 INFO:tasks.ceph.mon.a.trial057.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-12T09:10:08.314 INFO:tasks.ceph.mon.a.trial057.stderr:feature with important data. 2026-02-12T09:10:08.314 INFO:tasks.ceph.mon.a.trial057.stderr: 2026-02-12T09:10:09.076 INFO:tasks.ceph.mgr.x.trial057.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-12T09:10:09.076 INFO:tasks.ceph.mgr.x.trial057.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-12T09:10:09.077 INFO:tasks.ceph.mgr.x.trial057.stderr: from numpy import show_config as show_numpy_config 2026-02-12T09:10:09.200 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-12T09:10:09.200 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:09.200 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-12T09:10:09.265 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:09.265 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-12T09:10:09.330 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:09.330 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-12T09:10:09.395 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd new cea2c4e2-d503-4784-8674-bccec114485c 0 2026-02-12T09:10:09.508 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:09.505+0000 7fc6052cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:09.509 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:09.509+0000 7fc6052cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:09.547 INFO:teuthology.orchestra.run.trial057.stdout:0 2026-02-12T09:10:09.556 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd new cdd4974e-3d6a-4388-9db1-9b2e3b4c74d2 1 2026-02-12T09:10:09.630 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:09.627+0000 7f020f6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:09.631 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:09.631+0000 7f020f6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:09.667 INFO:teuthology.orchestra.run.trial057.stdout:1 2026-02-12T09:10:09.676 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd new 5c824285-031f-4919-898f-3e072fb6e1bb 2 2026-02-12T09:10:09.749 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:09.745+0000 7f20012cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:09.750 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:09.750+0000 7f20012cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:09.786 INFO:teuthology.orchestra.run.trial057.stdout:2 2026-02-12T09:10:09.795 DEBUG:teuthology.orchestra.run.trial057:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-12T09:10:09.824 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-12T09:10:09.824 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-12T09:10:09.866 INFO:tasks.ceph.osd.0:Started 2026-02-12T09:10:09.867 DEBUG:teuthology.orchestra.run.trial057:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-12T09:10:09.896 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-12T09:10:09.896 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-12T09:10:09.939 INFO:tasks.ceph.osd.1:Started 2026-02-12T09:10:09.939 DEBUG:teuthology.orchestra.run.trial057:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-12T09:10:09.967 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-12T09:10:09.968 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-12T09:10:10.010 INFO:tasks.ceph.osd.2:Started 2026-02-12T09:10:10.011 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T09:10:10.087 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:10.085+0000 7f8315e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:10.088 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:10.088+0000 7f8315e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:10.120 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:10:10.120 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":6,"fsid":"2d25ad73-62b5-4644-9a52-c68c542c153c","created":"2026-02-12T09:10:08.040640+0000","modified":"2026-02-12T09:10:09.787079+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T09:10:09.787079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"cea2c4e2-d503-4784-8674-bccec114485c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cdd4974e-3d6a-4388-9db1-9b2e3b4c74d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c824285-031f-4919-898f-3e072fb6e1bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:10:10.129 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-12T09:10:10.129 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-12T09:10:10.430 DEBUG:teuthology.orchestra.run.trial057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T09:10:10.499 INFO:teuthology.misc.health.trial057.stderr:2026-02-12T09:10:10.497+0000 7f3a7e56f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:10.501 INFO:teuthology.misc.health.trial057.stderr:2026-02-12T09:10:10.500+0000 7f3a7e56f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:10.533 INFO:teuthology.misc.health.trial057.stdout: 2026-02-12T09:10:10.533 INFO:teuthology.misc.health.trial057.stdout:{"epoch":6,"fsid":"2d25ad73-62b5-4644-9a52-c68c542c153c","created":"2026-02-12T09:10:08.040640+0000","modified":"2026-02-12T09:10:09.787079+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T09:10:09.787079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"cea2c4e2-d503-4784-8674-bccec114485c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cdd4974e-3d6a-4388-9db1-9b2e3b4c74d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c824285-031f-4919-898f-3e072fb6e1bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:10:10.540 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-12T09:10:11.472 INFO:tasks.ceph.mgr.x.trial057.stderr:2026-02-12T09:10:11.472+0000 7f248f19b640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-12T09:10:16.843 DEBUG:teuthology.orchestra.run.trial057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T09:10:16.924 INFO:teuthology.misc.health.trial057.stderr:2026-02-12T09:10:16.922+0000 7f61464cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:16.926 INFO:teuthology.misc.health.trial057.stderr:2026-02-12T09:10:16.926+0000 7f61464cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:17.060 INFO:teuthology.misc.health.trial057.stdout: 2026-02-12T09:10:17.061 INFO:teuthology.misc.health.trial057.stdout:{"epoch":7,"fsid":"2d25ad73-62b5-4644-9a52-c68c542c153c","created":"2026-02-12T09:10:08.040640+0000","modified":"2026-02-12T09:10:16.343037+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T09:10:09.787079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"cea2c4e2-d503-4784-8674-bccec114485c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"cdd4974e-3d6a-4388-9db1-9b2e3b4c74d2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5c824285-031f-4919-898f-3e072fb6e1bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:10:17.080 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-12T09:10:23.382 DEBUG:teuthology.orchestra.run.trial057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T09:10:23.449 INFO:teuthology.misc.health.trial057.stderr:2026-02-12T09:10:23.448+0000 7f7c54e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:23.451 INFO:teuthology.misc.health.trial057.stderr:2026-02-12T09:10:23.452+0000 7f7c54e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:23.559 INFO:teuthology.misc.health.trial057.stdout: 2026-02-12T09:10:23.559 INFO:teuthology.misc.health.trial057.stdout:{"epoch":13,"fsid":"2d25ad73-62b5-4644-9a52-c68c542c153c","created":"2026-02-12T09:10:08.040640+0000","modified":"2026-02-12T09:10:23.362323+0000","last_up_change":"2026-02-12T09:10:19.351409+0000","last_in_change":"2026-02-12T09:10:09.787079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T09:10:20.485339+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cea2c4e2-d503-4784-8674-bccec114485c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6808","nonce":1998955463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6809","nonce":1998955463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6804","nonce":1998955463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6805","nonce":1998955463}]},"public_addr":"10.20.193.57:6808/1998955463","cluster_addr":"10.20.193.57:6809/1998955463","heartbeat_back_addr":"10.20.193.57:6804/1998955463","heartbeat_front_addr":"10.20.193.57:6805/1998955463","state":["exists","up"]},{"osd":1,"uuid":"cdd4974e-3d6a-4388-9db1-9b2e3b4c74d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6814","nonce":330042033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6815","nonce":330042033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6818","nonce":330042033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6817","nonce":330042033}]},"public_addr":"10.20.193.57:6814/330042033","cluster_addr":"10.20.193.57:6815/330042033","heartbeat_back_addr":"10.20.193.57:6818/330042033","heartbeat_front_addr":"10.20.193.57:6817/330042033","state":["exists","up"]},{"osd":2,"uuid":"5c824285-031f-4919-898f-3e072fb6e1bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6802","nonce":922181814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6803","nonce":922181814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6807","nonce":922181814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6810","nonce":922181814}]},"public_addr":"10.20.193.57:6802/922181814","cluster_addr":"10.20.193.57:6803/922181814","heartbeat_back_addr":"10.20.193.57:6807/922181814","heartbeat_front_addr":"10.20.193.57:6810/922181814","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:10:23.568 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-12T09:10:23.568 INFO:tasks.ceph:Creating RBD pool 2026-02-12T09:10:23.568 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-12T09:10:23.643 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:23.640+0000 7fcd9e251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:23.644 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:23.645+0000 7fcd9e251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:24.368 INFO:teuthology.orchestra.run.trial057.stderr:pool 'rbd' created 2026-02-12T09:10:24.381 DEBUG:teuthology.orchestra.run.trial057:> rbd --cluster ceph pool init rbd 2026-02-12T09:10:24.426 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:24.426+0000 7f67b2e17240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:24.426 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:24.426+0000 7f67b2e17240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:24.427 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:24.428+0000 7f67b2e17240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:27.406 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-12T09:10:27.407 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-12T09:10:27.407 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-12T09:10:27.523 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:27.521+0000 7f68cfde4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:27.524 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:27.525+0000 7f68cfde4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:27.633 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:10:27.642 INFO:teuthology.orchestra.run.trial057.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-12T09:10:27.643 INFO:tasks.ceph_manager:config epoch is 1 2026-02-12T09:10:27.643 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-12T09:10:27.643 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-12T09:10:27.643 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-12T09:10:27.720 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:27.718+0000 7f220c6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:27.721 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:27.721+0000 7f220c6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:27.861 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:10:27.871 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6800","nonce":1135556484},{"type":"v1","addr":"10.20.193.57:6801","nonce":1135556484}]},"active_addr":"10.20.193.57:6801/1135556484","active_change":"2026-02-12T09:10:10.468507+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.57:0","nonce":2645208649}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.57:0","nonce":3209495328}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.57:0","nonce":1698689553}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.57:0","nonce":608766296}]}]} 2026-02-12T09:10:27.874 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-12T09:10:27.875 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-12T09:10:27.875 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T09:10:27.952 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:27.950+0000 7f35a12cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:27.953 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:27.954+0000 7f35a12cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.063 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:10:28.063 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":17,"fsid":"2d25ad73-62b5-4644-9a52-c68c542c153c","created":"2026-02-12T09:10:08.040640+0000","modified":"2026-02-12T09:10:27.378780+0000","last_up_change":"2026-02-12T09:10:19.351409+0000","last_in_change":"2026-02-12T09:10:09.787079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T09:10:20.485339+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T09:10:23.759958+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cea2c4e2-d503-4784-8674-bccec114485c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6808","nonce":1998955463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6809","nonce":1998955463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6804","nonce":1998955463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6805","nonce":1998955463}]},"public_addr":"10.20.193.57:6808/1998955463","cluster_addr":"10.20.193.57:6809/1998955463","heartbeat_back_addr":"10.20.193.57:6804/1998955463","heartbeat_front_addr":"10.20.193.57:6805/1998955463","state":["exists","up"]},{"osd":1,"uuid":"cdd4974e-3d6a-4388-9db1-9b2e3b4c74d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6814","nonce":330042033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6815","nonce":330042033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6818","nonce":330042033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6817","nonce":330042033}]},"public_addr":"10.20.193.57:6814/330042033","cluster_addr":"10.20.193.57:6815/330042033","heartbeat_back_addr":"10.20.193.57:6818/330042033","heartbeat_front_addr":"10.20.193.57:6817/330042033","state":["exists","up"]},{"osd":2,"uuid":"5c824285-031f-4919-898f-3e072fb6e1bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6802","nonce":922181814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6803","nonce":922181814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6807","nonce":922181814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6810","nonce":922181814}]},"public_addr":"10.20.193.57:6802/922181814","cluster_addr":"10.20.193.57:6803/922181814","heartbeat_back_addr":"10.20.193.57:6807/922181814","heartbeat_front_addr":"10.20.193.57:6810/922181814","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:10:28.073 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-12T09:10:28.073 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T09:10:28.148 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.147+0000 7fc27d446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.149 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.150+0000 7fc277fff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.258 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:10:28.258 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":17,"fsid":"2d25ad73-62b5-4644-9a52-c68c542c153c","created":"2026-02-12T09:10:08.040640+0000","modified":"2026-02-12T09:10:27.378780+0000","last_up_change":"2026-02-12T09:10:19.351409+0000","last_in_change":"2026-02-12T09:10:09.787079+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T09:10:20.485339+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T09:10:23.759958+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cea2c4e2-d503-4784-8674-bccec114485c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6808","nonce":1998955463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6809","nonce":1998955463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6804","nonce":1998955463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6805","nonce":1998955463}]},"public_addr":"10.20.193.57:6808/1998955463","cluster_addr":"10.20.193.57:6809/1998955463","heartbeat_back_addr":"10.20.193.57:6804/1998955463","heartbeat_front_addr":"10.20.193.57:6805/1998955463","state":["exists","up"]},{"osd":1,"uuid":"cdd4974e-3d6a-4388-9db1-9b2e3b4c74d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6814","nonce":330042033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6815","nonce":330042033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6818","nonce":330042033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6817","nonce":330042033}]},"public_addr":"10.20.193.57:6814/330042033","cluster_addr":"10.20.193.57:6815/330042033","heartbeat_back_addr":"10.20.193.57:6818/330042033","heartbeat_front_addr":"10.20.193.57:6817/330042033","state":["exists","up"]},{"osd":2,"uuid":"5c824285-031f-4919-898f-3e072fb6e1bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6802","nonce":922181814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6803","nonce":922181814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6807","nonce":922181814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6810","nonce":922181814}]},"public_addr":"10.20.193.57:6802/922181814","cluster_addr":"10.20.193.57:6803/922181814","heartbeat_back_addr":"10.20.193.57:6807/922181814","heartbeat_front_addr":"10.20.193.57:6810/922181814","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:10:28.269 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-12T09:10:28.269 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-12T09:10:28.270 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-12T09:10:28.350 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.349+0000 7fc09e4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.351 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.349+0000 7f48fe993640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.351 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.349+0000 7f0bbe30b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.352 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.353+0000 7f48fe993640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.352 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.353+0000 7fc09e4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.353 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.353+0000 7f0bbe30b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.367 INFO:teuthology.orchestra.run.trial057.stdout:38654705673 2026-02-12T09:10:28.368 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-12T09:10:28.368 INFO:teuthology.orchestra.run.trial057.stdout:42949672968 2026-02-12T09:10:28.369 INFO:teuthology.orchestra.run.trial057.stdout:42949672968 2026-02-12T09:10:28.370 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-12T09:10:28.370 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-12T09:10:28.487 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.485+0000 7f7a13acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.488 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.485+0000 7fbdddbb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.488 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.486+0000 7f7e8bd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.489 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.489+0000 7f7a13acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.489 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.489+0000 7fbdddbb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.489 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:28.490+0000 7f7e8bd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:28.604 INFO:teuthology.orchestra.run.trial057.stdout:42949672964 2026-02-12T09:10:28.606 INFO:teuthology.orchestra.run.trial057.stdout:42949672964 2026-02-12T09:10:28.607 INFO:teuthology.orchestra.run.trial057.stdout:38654705669 2026-02-12T09:10:28.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672964 for osd.1 2026-02-12T09:10:28.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672964 for osd.0 2026-02-12T09:10:28.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705669 for osd.2 2026-02-12T09:10:29.616 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-12T09:10:29.618 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-12T09:10:29.620 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-12T09:10:29.701 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:29.699+0000 7f6bf965a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:29.701 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:29.698+0000 7f25b0bdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:29.701 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:29.698+0000 7fbac150b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:29.702 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:29.703+0000 7fbac150b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:29.702 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:29.703+0000 7f6bf965a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:29.702 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:29.703+0000 7f25b0bdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:29.812 INFO:teuthology.orchestra.run.trial057.stdout:38654705674 2026-02-12T09:10:29.818 INFO:teuthology.orchestra.run.trial057.stdout:42949672968 2026-02-12T09:10:29.822 INFO:teuthology.orchestra.run.trial057.stdout:42949672968 2026-02-12T09:10:29.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705674 for osd.2 2026-02-12T09:10:29.824 DEBUG:teuthology.parallel:result is None 2026-02-12T09:10:29.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672968 for osd.1 2026-02-12T09:10:29.829 DEBUG:teuthology.parallel:result is None 2026-02-12T09:10:29.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672968 for osd.0 2026-02-12T09:10:29.833 DEBUG:teuthology.parallel:result is None 2026-02-12T09:10:29.833 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-12T09:10:29.833 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T09:10:29.950 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:29.948+0000 7f79f85c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:29.951 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:29.951+0000 7f79f7338640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:30.061 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:10:30.061 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-12T09:10:30.071 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-12T09:10:28.472426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2500,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497468,"statfs":{"total":160255967232,"available":160253407232,"internally_reserved":0,"allocated":2560000,"data_stored":2560000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.102680"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.394000+0000","last_change":"2026-02-12T09:10:27.397710+0000","last_active":"2026-02-12T09:10:27.394000+0000","last_peered":"2026-02-12T09:10:27.394000+0000","last_clean":"2026-02-12T09:10:27.394000+0000","last_became_active":"2026-02-12T09:10:25.411935+0000","last_became_peered":"2026-02-12T09:10:25.411935+0000","last_unstale":"2026-02-12T09:10:27.394000+0000","last_undegraded":"2026-02-12T09:10:27.394000+0000","last_fullsized":"2026-02-12T09:10:27.394000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.395106+0000","last_change":"2026-02-12T09:10:27.397294+0000","last_active":"2026-02-12T09:10:27.395106+0000","last_peered":"2026-02-12T09:10:27.395106+0000","last_clean":"2026-02-12T09:10:27.395106+0000","last_became_active":"2026-02-12T09:10:25.414681+0000","last_became_peered":"2026-02-12T09:10:25.414681+0000","last_unstale":"2026-02-12T09:10:27.395106+0000","last_undegraded":"2026-02-12T09:10:27.395106+0000","last_fullsized":"2026-02-12T09:10:27.395106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.395460+0000","last_change":"2026-02-12T09:10:27.399166+0000","last_active":"2026-02-12T09:10:27.395460+0000","last_peered":"2026-02-12T09:10:27.395460+0000","last_clean":"2026-02-12T09:10:27.395460+0000","last_became_active":"2026-02-12T09:10:25.415927+0000","last_became_peered":"2026-02-12T09:10:25.415927+0000","last_unstale":"2026-02-12T09:10:27.395460+0000","last_undegraded":"2026-02-12T09:10:27.395460+0000","last_fullsized":"2026-02-12T09:10:27.395460+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.395585+0000","last_change":"2026-02-12T09:10:27.399051+0000","last_active":"2026-02-12T09:10:27.395585+0000","last_peered":"2026-02-12T09:10:27.395585+0000","last_clean":"2026-02-12T09:10:27.395585+0000","last_became_active":"2026-02-12T09:10:25.409927+0000","last_became_peered":"2026-02-12T09:10:25.409927+0000","last_unstale":"2026-02-12T09:10:27.395585+0000","last_undegraded":"2026-02-12T09:10:27.395585+0000","last_fullsized":"2026-02-12T09:10:27.395585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.394499+0000","last_change":"2026-02-12T09:10:27.395740+0000","last_active":"2026-02-12T09:10:27.394499+0000","last_peered":"2026-02-12T09:10:27.394499+0000","last_clean":"2026-02-12T09:10:27.394499+0000","last_became_active":"2026-02-12T09:10:25.424781+0000","last_became_peered":"2026-02-12T09:10:25.424781+0000","last_unstale":"2026-02-12T09:10:27.394499+0000","last_undegraded":"2026-02-12T09:10:27.394499+0000","last_fullsized":"2026-02-12T09:10:27.394499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.392951+0000","last_change":"2026-02-12T09:10:27.394761+0000","last_active":"2026-02-12T09:10:27.392951+0000","last_peered":"2026-02-12T09:10:27.392951+0000","last_clean":"2026-02-12T09:10:27.392951+0000","last_became_active":"2026-02-12T09:10:25.425962+0000","last_became_peered":"2026-02-12T09:10:25.425962+0000","last_unstale":"2026-02-12T09:10:27.392951+0000","last_undegraded":"2026-02-12T09:10:27.392951+0000","last_fullsized":"2026-02-12T09:10:27.392951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.391365+0000","last_change":"2026-02-12T09:10:27.393627+0000","last_active":"2026-02-12T09:10:27.391365+0000","last_peered":"2026-02-12T09:10:27.391365+0000","last_clean":"2026-02-12T09:10:27.391365+0000","last_became_active":"2026-02-12T09:10:25.428342+0000","last_became_peered":"2026-02-12T09:10:25.428342+0000","last_unstale":"2026-02-12T09:10:27.391365+0000","last_undegraded":"2026-02-12T09:10:27.391365+0000","last_fullsized":"2026-02-12T09:10:27.391365+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.395113+0000","last_change":"2026-02-12T09:10:27.397289+0000","last_active":"2026-02-12T09:10:27.395113+0000","last_peered":"2026-02-12T09:10:27.395113+0000","last_clean":"2026-02-12T09:10:27.395113+0000","last_became_active":"2026-02-12T09:10:25.412111+0000","last_became_peered":"2026-02-12T09:10:25.412111+0000","last_unstale":"2026-02-12T09:10:27.395113+0000","last_undegraded":"2026-02-12T09:10:27.395113+0000","last_fullsized":"2026-02-12T09:10:27.395113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.393601+0000","last_change":"2026-02-12T09:10:22.390534+0000","last_active":"2026-02-12T09:10:27.393601+0000","last_peered":"2026-02-12T09:10:27.393601+0000","last_clean":"2026-02-12T09:10:27.393601+0000","last_became_active":"2026-02-12T09:10:22.383447+0000","last_became_peered":"2026-02-12T09:10:22.383447+0000","last_unstale":"2026-02-12T09:10:27.393601+0000","last_undegraded":"2026-02-12T09:10:27.393601+0000","last_fullsized":"2026-02-12T09:10:27.393601+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:21.353347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:21.353347+0000","last_clean_scrub_stamp":"2026-02-12T09:10:21.353347+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253366272,"internally_reserved":0,"allocated":2600960,"data_stored":2600960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1116,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165540,"statfs":{"total":53418655744,"available":53417512960,"internally_reserved":0,"allocated":1142784,"data_stored":1142784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1068,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165588,"statfs":{"total":53418655744,"available":53417562112,"internally_reserved":0,"allocated":1093632,"data_stored":1093632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":316,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166340,"statfs":{"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417553920,"internally_reserved":0,"allocated":1101824,"data_stored":1101824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T09:10:30.073 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T09:10:30.148 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:30.147+0000 7fc5dcb7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:30.149 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:30.150+0000 7fc5dcb7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:30.256 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:10:30.257 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-12T09:10:30.265 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-12T09:10:28.472426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2500,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497468,"statfs":{"total":160255967232,"available":160253407232,"internally_reserved":0,"allocated":2560000,"data_stored":2560000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.102680"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.394000+0000","last_change":"2026-02-12T09:10:27.397710+0000","last_active":"2026-02-12T09:10:27.394000+0000","last_peered":"2026-02-12T09:10:27.394000+0000","last_clean":"2026-02-12T09:10:27.394000+0000","last_became_active":"2026-02-12T09:10:25.411935+0000","last_became_peered":"2026-02-12T09:10:25.411935+0000","last_unstale":"2026-02-12T09:10:27.394000+0000","last_undegraded":"2026-02-12T09:10:27.394000+0000","last_fullsized":"2026-02-12T09:10:27.394000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.395106+0000","last_change":"2026-02-12T09:10:27.397294+0000","last_active":"2026-02-12T09:10:27.395106+0000","last_peered":"2026-02-12T09:10:27.395106+0000","last_clean":"2026-02-12T09:10:27.395106+0000","last_became_active":"2026-02-12T09:10:25.414681+0000","last_became_peered":"2026-02-12T09:10:25.414681+0000","last_unstale":"2026-02-12T09:10:27.395106+0000","last_undegraded":"2026-02-12T09:10:27.395106+0000","last_fullsized":"2026-02-12T09:10:27.395106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.395460+0000","last_change":"2026-02-12T09:10:27.399166+0000","last_active":"2026-02-12T09:10:27.395460+0000","last_peered":"2026-02-12T09:10:27.395460+0000","last_clean":"2026-02-12T09:10:27.395460+0000","last_became_active":"2026-02-12T09:10:25.415927+0000","last_became_peered":"2026-02-12T09:10:25.415927+0000","last_unstale":"2026-02-12T09:10:27.395460+0000","last_undegraded":"2026-02-12T09:10:27.395460+0000","last_fullsized":"2026-02-12T09:10:27.395460+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.395585+0000","last_change":"2026-02-12T09:10:27.399051+0000","last_active":"2026-02-12T09:10:27.395585+0000","last_peered":"2026-02-12T09:10:27.395585+0000","last_clean":"2026-02-12T09:10:27.395585+0000","last_became_active":"2026-02-12T09:10:25.409927+0000","last_became_peered":"2026-02-12T09:10:25.409927+0000","last_unstale":"2026-02-12T09:10:27.395585+0000","last_undegraded":"2026-02-12T09:10:27.395585+0000","last_fullsized":"2026-02-12T09:10:27.395585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.394499+0000","last_change":"2026-02-12T09:10:27.395740+0000","last_active":"2026-02-12T09:10:27.394499+0000","last_peered":"2026-02-12T09:10:27.394499+0000","last_clean":"2026-02-12T09:10:27.394499+0000","last_became_active":"2026-02-12T09:10:25.424781+0000","last_became_peered":"2026-02-12T09:10:25.424781+0000","last_unstale":"2026-02-12T09:10:27.394499+0000","last_undegraded":"2026-02-12T09:10:27.394499+0000","last_fullsized":"2026-02-12T09:10:27.394499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.392951+0000","last_change":"2026-02-12T09:10:27.394761+0000","last_active":"2026-02-12T09:10:27.392951+0000","last_peered":"2026-02-12T09:10:27.392951+0000","last_clean":"2026-02-12T09:10:27.392951+0000","last_became_active":"2026-02-12T09:10:25.425962+0000","last_became_peered":"2026-02-12T09:10:25.425962+0000","last_unstale":"2026-02-12T09:10:27.392951+0000","last_undegraded":"2026-02-12T09:10:27.392951+0000","last_fullsized":"2026-02-12T09:10:27.392951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.391365+0000","last_change":"2026-02-12T09:10:27.393627+0000","last_active":"2026-02-12T09:10:27.391365+0000","last_peered":"2026-02-12T09:10:27.391365+0000","last_clean":"2026-02-12T09:10:27.391365+0000","last_became_active":"2026-02-12T09:10:25.428342+0000","last_became_peered":"2026-02-12T09:10:25.428342+0000","last_unstale":"2026-02-12T09:10:27.391365+0000","last_undegraded":"2026-02-12T09:10:27.391365+0000","last_fullsized":"2026-02-12T09:10:27.391365+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.395113+0000","last_change":"2026-02-12T09:10:27.397289+0000","last_active":"2026-02-12T09:10:27.395113+0000","last_peered":"2026-02-12T09:10:27.395113+0000","last_clean":"2026-02-12T09:10:27.395113+0000","last_became_active":"2026-02-12T09:10:25.412111+0000","last_became_peered":"2026-02-12T09:10:25.412111+0000","last_unstale":"2026-02-12T09:10:27.395113+0000","last_undegraded":"2026-02-12T09:10:27.395113+0000","last_fullsized":"2026-02-12T09:10:27.395113+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:24.366757+0000","last_clean_scrub_stamp":"2026-02-12T09:10:24.366757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T09:10:27.393601+0000","last_change":"2026-02-12T09:10:22.390534+0000","last_active":"2026-02-12T09:10:27.393601+0000","last_peered":"2026-02-12T09:10:27.393601+0000","last_clean":"2026-02-12T09:10:27.393601+0000","last_became_active":"2026-02-12T09:10:22.383447+0000","last_became_peered":"2026-02-12T09:10:22.383447+0000","last_unstale":"2026-02-12T09:10:27.393601+0000","last_undegraded":"2026-02-12T09:10:27.393601+0000","last_fullsized":"2026-02-12T09:10:27.393601+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:10:21.353347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:10:21.353347+0000","last_clean_scrub_stamp":"2026-02-12T09:10:21.353347+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253366272,"internally_reserved":0,"allocated":2600960,"data_stored":2600960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1116,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165540,"statfs":{"total":53418655744,"available":53417512960,"internally_reserved":0,"allocated":1142784,"data_stored":1142784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1068,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165588,"statfs":{"total":53418655744,"available":53417562112,"internally_reserved":0,"allocated":1093632,"data_stored":1093632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":316,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166340,"statfs":{"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417553920,"internally_reserved":0,"allocated":1101824,"data_stored":1101824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T09:10:30.266 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-12T09:10:30.266 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-12T09:10:30.266 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-12T09:10:30.266 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-12T09:10:30.341 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:30.341+0000 7f797bacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:30.343 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:30.344+0000 7f797a842640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:30.480 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-12T09:10:30.480 INFO:teuthology.orchestra.run.trial057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-12T09:10:30.490 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-12T09:10:30.491 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-12T09:10:30.564 DEBUG:teuthology.orchestra.run.trial057:> echo 'StrictHostKeyChecking no 2026-02-12T09:10:30.564 DEBUG:teuthology.orchestra.run.trial057:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-12T09:10:30.580 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQC2KIYxN1eAvAz8WUkIApRfK9aUQ1WRsgPDq7ECniSaPSelK/tO4+wXv3ku8fexcL8g/CkDFf36T48Ltww+0jqqSfEdC35u2+M/rTfRxAVOILOsRaKETGUcdxKU9czAiMyPpa26b2ipvyXVgkriII72CTK5tAPRdLD3paRD7TVCgbjTb6az14/F+VPk5KJDJWFwvmTLpV5v95TobVZk8LdCnF2XsPGCm2z+Tk6ITanu7g3mZO8LsrmRsD8iCqPOJK/SR4dI2GP/lvUCi93mTWoAOqGMZE0gVGDpLdNNpnLiD8xdxHVfgTyVo90cYrnHzQn2KYPN/zQ5z/9Pmw04kSVN 2026-02-12T09:10:30.580 INFO:teuthology.task.ssh_keys:pushing keys to trial057.front.sepia.ceph.com for ubuntu 2026-02-12T09:10:30.580 DEBUG:teuthology.orchestra.run.trial057:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-12T09:10:30.636 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:30.636 DEBUG:teuthology.orchestra.run.trial057:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-12T09:10:30.636 DEBUG:teuthology.orchestra.run.trial057:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-12T09:10:30.693 DEBUG:teuthology.orchestra.run.trial057:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-12T09:10:30.750 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:30.750 DEBUG:teuthology.orchestra.run.trial057:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-12T09:10:30.806 DEBUG:teuthology.orchestra.run.trial057:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-12_09:10:30:806494 2026-02-12T09:10:30.863 INFO:teuthology.orchestra.run.trial057.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-12_09:10:30:806494' 2026-02-12T09:10:30.863 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-12T09:10:30.863 DEBUG:teuthology.orchestra.run.trial057:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-12T09:10:30.919 INFO:teuthology.run_tasks:Running task radosbench... 2026-02-12T09:10:30.937 INFO:tasks.radosbench:Beginning radosbench... 2026-02-12T09:10:30.937 INFO:tasks.radosbench:omap writes 2026-02-12T09:10:30.937 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2026-02-12T09:10:30.938 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2026-02-12T09:10:31.038 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:31.035+0000 7f98c41d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:31.039 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:31.040+0000 7f98c2f4a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:31.388 INFO:teuthology.orchestra.run.trial057.stderr:pool 'unique_pool_0' created 2026-02-12T09:10:31.402 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2026-02-12T09:10:31.485 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:31.480+0000 7ff2e3811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:31.486 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:10:31.485+0000 7ff2e3811640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:10:32.389 INFO:teuthology.orchestra.run.trial057.stderr:enabled application 'rados' on pool 'unique_pool_0' 2026-02-12T09:10:33.406 DEBUG:teuthology.orchestra.run.trial057:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -t 16 -b 4096 --object-size 4096 -p unique_pool_0 bench 300 write --write-omap' 2026-02-12T09:10:33.408 DEBUG:teuthology.run_tasks:Unwinding manager radosbench 2026-02-12T09:10:33.423 INFO:tasks.radosbench:joining radosbench (timing out after 9300s) 2026-02-12T09:10:33.423 INFO:teuthology.orchestra.run:waiting for 9300 2026-02-12T09:10:33.455 INFO:tasks.radosbench.radosbench.0.trial057.stdout:hints = 1 2026-02-12T09:10:33.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 300 seconds or 0 objects 2026-02-12T09:10:33.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Object prefix: benchmark_data_trial057_23905 2026-02-12T09:10:33.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:10:33.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 0 0 0 0 0 0 - 0 2026-02-12T09:10:34.455 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 1 16 348 332 1.29675 1.29688 0.0264939 0.0454492 2026-02-12T09:10:35.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 2 16 667 651 1.27137 1.24609 0.0929593 0.0483542 2026-02-12T09:10:36.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 3 16 1028 1012 1.3176 1.41016 0.0366578 0.0470844 2026-02-12T09:10:37.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 4 16 1297 1281 1.25087 1.05078 0.0278749 0.0489891 2026-02-12T09:10:38.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 5 16 1624 1608 1.25615 1.27734 0.039649 0.0494792 2026-02-12T09:10:39.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 6 16 1990 1974 1.28506 1.42969 0.0194975 0.0484041 2026-02-12T09:10:40.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 7 16 2346 2330 1.30012 1.39062 0.0535726 0.0478848 2026-02-12T09:10:41.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 8 16 2705 2689 1.31289 1.40234 0.0457382 0.0474509 2026-02-12T09:10:42.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 9 16 3063 3047 1.32238 1.39844 0.0146461 0.0470455 2026-02-12T09:10:43.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 10 16 3415 3399 1.32763 1.375 0.0167501 0.0469552 2026-02-12T09:10:44.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 11 16 3785 3769 1.33832 1.44531 0.0300796 0.0465858 2026-02-12T09:10:45.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 12 16 4148 4132 1.34495 1.41797 0.0301348 0.0463634 2026-02-12T09:10:46.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 13 16 4500 4484 1.34725 1.375 0.0532067 0.0463062 2026-02-12T09:10:47.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 14 16 4858 4842 1.3509 1.39844 0.0403055 0.0462016 2026-02-12T09:10:48.456 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 15 16 5215 5199 1.3538 1.39453 0.0278481 0.0460571 2026-02-12T09:10:49.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 16 16 5579 5563 1.35805 1.42188 0.0866632 0.0459493 2026-02-12T09:10:50.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 17 16 5942 5926 1.36156 1.41797 0.0303271 0.0458333 2026-02-12T09:10:51.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 18 16 6318 6302 1.36751 1.46875 0.0297083 0.0456331 2026-02-12T09:10:52.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 19 16 6678 6662 1.36954 1.40625 0.0367281 0.0455878 2026-02-12T09:10:53.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:10:53.458519+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0456342 2026-02-12T09:10:53.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:10:53.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 20 16 7015 6999 1.36688 1.31641 0.0213356 0.0456342 2026-02-12T09:10:54.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 21 16 7380 7364 1.36968 1.42578 0.0368443 0.0455873 2026-02-12T09:10:55.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 22 16 7739 7723 1.37116 1.40234 0.0549581 0.0455134 2026-02-12T09:10:56.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 23 16 8103 8087 1.37336 1.42188 0.0351594 0.0454577 2026-02-12T09:10:57.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 24 16 8457 8441 1.37375 1.38281 0.0271864 0.0454264 2026-02-12T09:10:58.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 25 16 8809 8793 1.3738 1.375 0.0199934 0.0454409 2026-02-12T09:10:59.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 26 16 9161 9145 1.37384 1.375 0.0200974 0.0454502 2026-02-12T09:11:00.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 27 16 9508 9492 1.37315 1.35547 0.0555328 0.0454674 2026-02-12T09:11:01.457 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 28 16 9846 9830 1.37126 1.32031 0.0853275 0.0455156 2026-02-12T09:11:02.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 29 16 10173 10157 1.36802 1.27734 0.045707 0.0456534 2026-02-12T09:11:03.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 30 16 10493 10477 1.36408 1.25 0.0263852 0.0457535 2026-02-12T09:11:04.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 31 16 10801 10785 1.35889 1.20312 0.0714575 0.0459388 2026-02-12T09:11:05.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 32 16 11104 11088 1.35341 1.18359 0.0259633 0.0461255 2026-02-12T09:11:06.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 33 16 11431 11415 1.3511 1.27734 0.0435665 0.0462109 2026-02-12T09:11:07.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 34 16 11737 11721 1.34651 1.19531 0.0425081 0.0463664 2026-02-12T09:11:08.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 35 16 12060 12044 1.34409 1.26172 0.0422134 0.0464568 2026-02-12T09:11:09.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 36 16 12373 12357 1.34071 1.22266 0.0421709 0.0465785 2026-02-12T09:11:10.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 37 16 12675 12659 1.33636 1.17969 0.0322134 0.0467165 2026-02-12T09:11:11.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 38 16 12974 12958 1.33193 1.16797 0.0464376 0.0468769 2026-02-12T09:11:12.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 39 16 13282 13266 1.32862 1.20312 0.0713886 0.046999 2026-02-12T09:11:13.458 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:11:13.460105+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0471099 2026-02-12T09:11:13.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:11:13.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 40 16 13594 13578 1.32587 1.21875 0.0592173 0.0471099 2026-02-12T09:11:14.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 41 16 13903 13887 1.32297 1.20703 0.0187202 0.0472058 2026-02-12T09:11:15.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 42 16 14196 14180 1.31872 1.14453 0.0284961 0.0473601 2026-02-12T09:11:16.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 43 16 14509 14493 1.31648 1.22266 0.0440188 0.0474422 2026-02-12T09:11:17.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 44 16 14802 14786 1.31257 1.14453 0.126666 0.0475727 2026-02-12T09:11:18.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 45 16 15095 15079 1.30884 1.14453 0.0525781 0.0477172 2026-02-12T09:11:19.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 46 16 15397 15381 1.30603 1.17969 0.0422366 0.0478234 2026-02-12T09:11:20.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 47 16 15692 15676 1.30276 1.15234 0.0513921 0.0479405 2026-02-12T09:11:21.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 48 16 15987 15971 1.29962 1.15234 0.0303098 0.0480471 2026-02-12T09:11:22.459 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 49 16 16246 16230 1.29374 1.01172 0.0701867 0.0482713 2026-02-12T09:11:23.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 50 16 16534 16518 1.29037 1.125 0.0843727 0.0484037 2026-02-12T09:11:24.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 51 16 16823 16807 1.2872 1.12891 0.0795009 0.0485177 2026-02-12T09:11:25.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 52 16 17102 17086 1.2834 1.08984 0.0373616 0.0486682 2026-02-12T09:11:26.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 53 16 17387 17371 1.28019 1.11328 0.0555014 0.0487931 2026-02-12T09:11:27.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 54 16 17662 17646 1.27637 1.07422 0.0233771 0.0489258 2026-02-12T09:11:28.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 55 16 17954 17938 1.2739 1.14062 0.0392321 0.0490316 2026-02-12T09:11:29.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 56 16 18217 18201 1.2695 1.02734 0.141114 0.049189 2026-02-12T09:11:30.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 57 16 18505 18489 1.26696 1.125 0.129857 0.0492852 2026-02-12T09:11:31.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 58 16 18798 18782 1.26485 1.14453 0.0282186 0.0493877 2026-02-12T09:11:32.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 59 16 19099 19083 1.26334 1.17578 0.0411712 0.0494462 2026-02-12T09:11:33.460 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:11:33.461690+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0495662 2026-02-12T09:11:33.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:11:33.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 60 16 19371 19355 1.25999 1.0625 0.0438139 0.0495662 2026-02-12T09:11:34.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 61 16 19669 19653 1.25842 1.16406 0.0234626 0.0496344 2026-02-12T09:11:35.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 62 16 19954 19938 1.25607 1.11328 0.0481586 0.049729 2026-02-12T09:11:36.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 63 16 20234 20218 1.2535 1.09375 0.0749762 0.049835 2026-02-12T09:11:37.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 64 16 20511 20495 1.25082 1.08203 0.0452257 0.0499462 2026-02-12T09:11:38.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 65 16 20788 20772 1.24822 1.08203 0.0435581 0.0500399 2026-02-12T09:11:39.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 66 16 21072 21056 1.24611 1.10938 0.0561245 0.0501256 2026-02-12T09:11:40.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 67 16 21335 21319 1.24285 1.02734 0.0348364 0.0502544 2026-02-12T09:11:41.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 68 16 21626 21610 1.24128 1.13672 0.0450144 0.0503265 2026-02-12T09:11:42.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 69 16 21909 21893 1.23931 1.10547 0.053913 0.0504024 2026-02-12T09:11:43.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 70 16 22190 22174 1.23729 1.09766 0.0841046 0.0504841 2026-02-12T09:11:44.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 71 16 22465 22449 1.23499 1.07422 0.106187 0.0505763 2026-02-12T09:11:45.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 72 16 22731 22715 1.23227 1.03906 0.0554971 0.0506738 2026-02-12T09:11:46.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 73 16 23012 22996 1.23042 1.09766 0.063598 0.0507661 2026-02-12T09:11:47.461 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 74 16 23287 23271 1.22831 1.07422 0.0770519 0.0508624 2026-02-12T09:11:48.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 75 16 23556 23540 1.22594 1.05078 0.0391129 0.0509596 2026-02-12T09:11:49.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 76 16 23816 23800 1.22318 1.01562 0.0522615 0.0510614 2026-02-12T09:11:50.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 77 16 24096 24080 1.22149 1.09375 0.0420176 0.0511408 2026-02-12T09:11:51.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 78 16 24362 24346 1.21915 1.03906 0.0310448 0.0512352 2026-02-12T09:11:52.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 79 16 24609 24593 1.21593 0.964844 0.100489 0.0513738 2026-02-12T09:11:53.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:11:53.463256+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0514876 2026-02-12T09:11:53.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:11:53.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 80 16 24866 24850 1.21328 1.00391 0.0588036 0.0514876 2026-02-12T09:11:54.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 81 16 25139 25123 1.21147 1.06641 0.0510384 0.0515506 2026-02-12T09:11:55.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 82 16 25415 25399 1.20984 1.07812 0.109063 0.0516367 2026-02-12T09:11:56.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 83 16 25668 25652 1.20717 0.988281 0.0498753 0.0517373 2026-02-12T09:11:57.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 84 16 25947 25931 1.20577 1.08984 0.045029 0.0518082 2026-02-12T09:11:58.462 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 85 16 26224 26208 1.20432 1.08203 0.0457375 0.0518751 2026-02-12T09:11:59.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 86 16 26494 26478 1.20258 1.05469 0.150967 0.051947 2026-02-12T09:12:00.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 87 16 26752 26736 1.20034 1.00781 0.220104 0.0520407 2026-02-12T09:12:01.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 88 16 26998 26982 1.19762 0.960938 0.0630794 0.052149 2026-02-12T09:12:02.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 89 16 27269 27253 1.19605 1.05859 0.0431192 0.0522302 2026-02-12T09:12:03.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 90 16 27527 27511 1.19396 1.00781 0.0998935 0.0523213 2026-02-12T09:12:04.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 91 16 27780 27764 1.1917 0.988281 0.0643982 0.0524204 2026-02-12T09:12:05.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 92 16 28044 28028 1.18995 1.03125 0.0615265 0.0524954 2026-02-12T09:12:06.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 93 16 28304 28288 1.18808 1.01562 0.083819 0.052578 2026-02-12T09:12:07.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 94 16 28568 28552 1.18641 1.03125 0.0485631 0.052654 2026-02-12T09:12:08.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 95 16 28831 28815 1.18473 1.02734 0.0351399 0.0527256 2026-02-12T09:12:09.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 96 16 29091 29075 1.18297 1.01562 0.0503277 0.0528109 2026-02-12T09:12:10.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 97 16 29350 29334 1.18121 1.01172 0.080059 0.0528889 2026-02-12T09:12:11.463 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 98 16 29607 29591 1.1794 1.00391 0.051943 0.0529656 2026-02-12T09:12:12.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 99 16 29860 29844 1.17746 0.988281 0.0193007 0.0530526 2026-02-12T09:12:13.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:12:13.464797+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0531421 2026-02-12T09:12:13.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:12:13.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 100 16 30113 30097 1.17557 0.988281 0.0530023 0.0531421 2026-02-12T09:12:14.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 101 16 30362 30346 1.17356 0.972656 0.0603296 0.0532196 2026-02-12T09:12:15.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 102 16 30632 30616 1.1724 1.05469 0.0444366 0.053285 2026-02-12T09:12:16.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 103 16 30894 30878 1.17095 1.02344 0.0917159 0.0533535 2026-02-12T09:12:17.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 104 16 31149 31133 1.16927 0.996094 0.0823185 0.0534252 2026-02-12T09:12:18.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 105 16 31418 31402 1.16814 1.05078 0.0288681 0.053486 2026-02-12T09:12:19.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 106 16 31685 31669 1.16696 1.04297 0.0935839 0.0535391 2026-02-12T09:12:20.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 107 16 31946 31930 1.16558 1.01953 0.0399091 0.0535992 2026-02-12T09:12:21.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 108 16 32199 32183 1.16393 0.988281 0.0473527 0.0536665 2026-02-12T09:12:22.464 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 109 16 32435 32419 1.16171 0.921875 0.0417817 0.0537687 2026-02-12T09:12:23.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 110 16 32678 32662 1.15978 0.949219 0.0538279 0.053868 2026-02-12T09:12:24.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 111 16 32931 32915 1.15824 0.988281 0.0762256 0.0539391 2026-02-12T09:12:25.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 112 16 33182 33166 1.15665 0.980469 0.051741 0.0540074 2026-02-12T09:12:26.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 113 16 33443 33427 1.15543 1.01953 0.0727714 0.0540696 2026-02-12T09:12:27.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 114 16 33700 33684 1.1541 1.00391 0.0285836 0.0541394 2026-02-12T09:12:28.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 115 16 33945 33929 1.15239 0.957031 0.0749291 0.0542149 2026-02-12T09:12:29.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 116 16 34197 34181 1.15094 0.984375 0.0512011 0.0542773 2026-02-12T09:12:30.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 117 16 34449 34433 1.14952 0.984375 0.127305 0.0543492 2026-02-12T09:12:31.465 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 118 16 34689 34673 1.14772 0.9375 0.0522829 0.0544347 2026-02-12T09:12:32.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 119 16 34923 34907 1.14575 0.914062 0.0411481 0.054518 2026-02-12T09:12:33.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:12:33.466712+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0545904 2026-02-12T09:12:33.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:12:33.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 120 16 35176 35160 1.14444 0.988281 0.0586612 0.0545904 2026-02-12T09:12:34.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 121 16 35427 35411 1.14308 0.980469 0.103783 0.0546514 2026-02-12T09:12:35.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 122 16 35658 35642 1.14111 0.902344 0.0858688 0.0547477 2026-02-12T09:12:36.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 123 16 35891 35875 1.13923 0.910156 0.14724 0.0548366 2026-02-12T09:12:37.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 124 16 36136 36120 1.13776 0.957031 0.117154 0.0549153 2026-02-12T09:12:38.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 125 16 36383 36367 1.13638 0.964844 0.0674689 0.0549843 2026-02-12T09:12:39.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 126 16 36624 36608 1.13483 0.941406 0.0981992 0.0550445 2026-02-12T09:12:40.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 127 16 36864 36848 1.13327 0.9375 0.0413334 0.0551328 2026-02-12T09:12:41.466 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 128 16 37093 37077 1.13141 0.894531 0.0503423 0.0552186 2026-02-12T09:12:42.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 129 16 37340 37324 1.13012 0.964844 0.054192 0.0552805 2026-02-12T09:12:43.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 130 16 37590 37574 1.12893 0.976562 0.0429362 0.0553433 2026-02-12T09:12:44.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 131 16 37839 37823 1.12774 0.972656 0.0503813 0.0554021 2026-02-12T09:12:45.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 132 16 38094 38078 1.12674 0.996094 0.0637193 0.0554468 2026-02-12T09:12:46.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 133 16 38351 38335 1.12582 1.00391 0.0450098 0.0554953 2026-02-12T09:12:47.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 134 16 38596 38580 1.12456 0.957031 0.0564899 0.0555506 2026-02-12T09:12:48.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 135 16 38841 38825 1.12332 0.957031 0.0623574 0.055617 2026-02-12T09:12:49.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 136 16 39079 39063 1.12189 0.929688 0.0609676 0.0556891 2026-02-12T09:12:50.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 137 16 39280 39264 1.11943 0.785156 0.113904 0.0558064 2026-02-12T09:12:51.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 138 16 39521 39505 1.11814 0.941406 0.0668242 0.0558739 2026-02-12T09:12:52.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 139 16 39759 39743 1.11679 0.929688 0.0312276 0.0559428 2026-02-12T09:12:53.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:12:53.468260+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0559923 2026-02-12T09:12:53.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:12:53.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 140 16 40014 39998 1.11593 0.996094 0.0930905 0.0559923 2026-02-12T09:12:54.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 141 16 40259 40243 1.1148 0.957031 0.0274456 0.0560397 2026-02-12T09:12:55.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 142 16 40491 40475 1.11333 0.90625 0.0368495 0.0561152 2026-02-12T09:12:56.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 143 16 40741 40725 1.11237 0.976562 0.0568081 0.0561587 2026-02-12T09:12:57.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 144 16 40976 40960 1.11102 0.917969 0.0691812 0.0562384 2026-02-12T09:12:58.467 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 145 16 41200 41184 1.10939 0.875 0.0518968 0.0563164 2026-02-12T09:12:59.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 146 16 41442 41426 1.10827 0.945312 0.0842942 0.0563772 2026-02-12T09:13:00.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 147 16 41676 41660 1.10695 0.914062 0.0397163 0.0564475 2026-02-12T09:13:01.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 148 16 41910 41894 1.10564 0.914062 0.0499654 0.0565081 2026-02-12T09:13:02.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 149 16 42151 42135 1.10454 0.941406 0.0275698 0.0565654 2026-02-12T09:13:03.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 150 16 42397 42381 1.10358 0.960938 0.0398869 0.0566164 2026-02-12T09:13:04.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 151 16 42629 42613 1.10228 0.90625 0.0986519 0.0566815 2026-02-12T09:13:05.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 152 16 42863 42847 1.10104 0.914062 0.0437753 0.0567355 2026-02-12T09:13:06.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 153 16 43103 43087 1.09997 0.9375 0.0527966 0.0568019 2026-02-12T09:13:07.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 154 16 43343 43327 1.09891 0.9375 0.045267 0.0568569 2026-02-12T09:13:08.468 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 155 16 43581 43565 1.09782 0.929688 0.107148 0.0569129 2026-02-12T09:13:09.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 156 16 43813 43797 1.09659 0.90625 0.0471773 0.0569796 2026-02-12T09:13:10.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 157 16 44043 44027 1.09533 0.898438 0.0301146 0.0570392 2026-02-12T09:13:11.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 158 16 44278 44262 1.09421 0.917969 0.0482231 0.0571024 2026-02-12T09:13:12.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 159 16 44506 44490 1.09293 0.890625 0.0457459 0.0571669 2026-02-12T09:13:13.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:13:13.469764+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0572272 2026-02-12T09:13:13.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:13:13.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 160 16 44740 44724 1.09181 0.914062 0.0612242 0.0572272 2026-02-12T09:13:14.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 161 16 44979 44963 1.09082 0.933594 0.156098 0.0572809 2026-02-12T09:13:15.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 162 16 45211 45195 1.08968 0.90625 0.0508987 0.0573367 2026-02-12T09:13:16.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 163 16 45444 45428 1.08858 0.910156 0.081315 0.057396 2026-02-12T09:13:17.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 164 16 45680 45664 1.08757 0.921875 0.0369327 0.0574506 2026-02-12T09:13:18.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 165 16 45910 45894 1.08642 0.898438 0.110316 0.057508 2026-02-12T09:13:19.469 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 166 16 46126 46110 1.08496 0.84375 0.0309632 0.0575863 2026-02-12T09:13:20.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 167 16 46336 46320 1.08337 0.820312 0.139635 0.0576745 2026-02-12T09:13:21.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 168 16 46552 46536 1.08194 0.84375 0.0865615 0.0577502 2026-02-12T09:13:22.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 169 16 46755 46739 1.08023 0.792969 0.0833896 0.057841 2026-02-12T09:13:23.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 170 16 46990 46974 1.07928 0.917969 0.0800605 0.0578924 2026-02-12T09:13:24.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 171 16 47216 47200 1.07813 0.882812 0.0746502 0.0579557 2026-02-12T09:13:25.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 172 16 47437 47421 1.07688 0.863281 0.0428467 0.0580186 2026-02-12T09:13:26.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 173 16 47662 47646 1.07574 0.878906 0.0428539 0.058081 2026-02-12T09:13:27.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 174 16 47896 47880 1.07481 0.914062 0.108354 0.0581342 2026-02-12T09:13:28.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 175 16 48128 48112 1.07384 0.90625 0.0679814 0.0581786 2026-02-12T09:13:29.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 176 16 48366 48350 1.07302 0.929688 0.0474782 0.0582298 2026-02-12T09:13:30.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 177 16 48562 48546 1.07129 0.765625 0.0919886 0.0583249 2026-02-12T09:13:31.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 178 16 48797 48781 1.07042 0.917969 0.0778662 0.0583696 2026-02-12T09:13:32.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 179 16 49011 48995 1.06911 0.835938 0.0822065 0.0584349 2026-02-12T09:13:33.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:13:33.471328+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0584935 2026-02-12T09:13:33.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:13:33.470 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 180 16 49241 49225 1.06817 0.898438 0.0988643 0.0584935 2026-02-12T09:13:34.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 181 16 49482 49466 1.06746 0.941406 0.0837516 0.0585358 2026-02-12T09:13:35.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 182 16 49721 49705 1.06673 0.933594 0.0873008 0.0585748 2026-02-12T09:13:36.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 183 16 49951 49935 1.06581 0.898438 0.0310112 0.0586236 2026-02-12T09:13:37.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 184 16 50171 50155 1.06469 0.859375 0.0676925 0.0586873 2026-02-12T09:13:38.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 185 16 50393 50377 1.06362 0.867188 0.0327681 0.0587465 2026-02-12T09:13:39.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 186 16 50613 50597 1.06252 0.859375 0.0315152 0.0588048 2026-02-12T09:13:40.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 187 16 50821 50805 1.06118 0.8125 0.0441481 0.05887 2026-02-12T09:13:41.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 188 16 51041 51025 1.06011 0.859375 0.0860779 0.0589351 2026-02-12T09:13:42.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 189 16 51251 51235 1.05884 0.820312 0.0606888 0.0590072 2026-02-12T09:13:43.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 190 16 51458 51442 1.05752 0.808594 0.0808174 0.0590839 2026-02-12T09:13:44.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 191 16 51683 51667 1.05659 0.878906 0.153267 0.0591293 2026-02-12T09:13:45.471 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 192 16 51901 51885 1.05552 0.851562 0.0289683 0.0591942 2026-02-12T09:13:46.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 193 16 52127 52111 1.05462 0.882812 0.034676 0.0592474 2026-02-12T09:13:47.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 194 16 52357 52341 1.05382 0.898438 0.0475333 0.0592919 2026-02-12T09:13:48.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 195 16 52571 52555 1.0527 0.835938 0.0364815 0.0593551 2026-02-12T09:13:49.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 196 16 52796 52780 1.05181 0.878906 0.0614005 0.0594025 2026-02-12T09:13:50.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 197 16 53004 52988 1.0506 0.8125 0.0427041 0.0594587 2026-02-12T09:13:51.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 198 16 53205 53189 1.04926 0.785156 0.0650174 0.0595505 2026-02-12T09:13:52.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 199 16 53421 53405 1.04822 0.84375 0.119116 0.0596103 2026-02-12T09:13:53.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:13:53.472864+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.059655 2026-02-12T09:13:53.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:13:53.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 200 16 53645 53629 1.04736 0.875 0.115922 0.059655 2026-02-12T09:13:54.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 201 16 53859 53843 1.04631 0.835938 0.0842296 0.05972 2026-02-12T09:13:55.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 202 16 54080 54064 1.0454 0.863281 0.0505193 0.05977 2026-02-12T09:13:56.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 203 16 54304 54288 1.04456 0.875 0.104153 0.0598209 2026-02-12T09:13:57.472 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 204 16 54525 54509 1.04367 0.863281 0.0570154 0.0598674 2026-02-12T09:13:58.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 205 16 54742 54726 1.04271 0.847656 0.0747676 0.0599226 2026-02-12T09:13:59.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 206 16 54968 54952 1.04194 0.882812 0.0666313 0.059967 2026-02-12T09:14:00.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 207 16 55184 55168 1.04098 0.84375 0.0578437 0.0600137 2026-02-12T09:14:01.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 208 16 55396 55380 1.03996 0.828125 0.0838297 0.0600804 2026-02-12T09:14:02.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 209 16 55618 55602 1.03913 0.867188 0.0513706 0.060127 2026-02-12T09:14:03.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 210 16 55836 55820 1.03824 0.851562 0.0769834 0.0601756 2026-02-12T09:14:04.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 211 16 56056 56040 1.03739 0.859375 0.0579175 0.0602333 2026-02-12T09:14:05.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 212 16 56261 56245 1.03627 0.800781 0.0596527 0.0602948 2026-02-12T09:14:06.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 213 16 56473 56457 1.03529 0.828125 0.0777528 0.0603556 2026-02-12T09:14:07.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 214 16 56692 56676 1.03445 0.855469 0.0642942 0.0604031 2026-02-12T09:14:08.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 215 16 56895 56879 1.03333 0.792969 0.0758364 0.0604703 2026-02-12T09:14:09.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 216 16 57108 57092 1.0324 0.832031 0.0339457 0.0605191 2026-02-12T09:14:10.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 217 16 57317 57301 1.0314 0.816406 0.0475187 0.0605762 2026-02-12T09:14:11.473 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 218 16 57516 57500 1.03024 0.777344 0.0277402 0.0606457 2026-02-12T09:14:12.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 219 16 57713 57697 1.02905 0.769531 0.0497592 0.060718 2026-02-12T09:14:13.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:14:13.474416+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0607771 2026-02-12T09:14:13.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:14:13.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 220 16 57916 57900 1.02797 0.792969 0.0615415 0.0607771 2026-02-12T09:14:14.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 221 16 58146 58130 1.02739 0.898438 0.0984762 0.0608182 2026-02-12T09:14:15.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 222 16 58350 58334 1.02635 0.796875 0.0579628 0.0608737 2026-02-12T09:14:16.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 223 16 58542 58526 1.02511 0.75 0.0452232 0.0609258 2026-02-12T09:14:17.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 224 16 58740 58724 1.02398 0.773438 0.0689181 0.0610069 2026-02-12T09:14:18.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 225 16 58960 58944 1.02325 0.859375 0.0414901 0.0610649 2026-02-12T09:14:19.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 226 16 59184 59168 1.0226 0.875 0.0558347 0.061105 2026-02-12T09:14:20.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 227 16 59403 59387 1.02186 0.855469 0.0542393 0.0611508 2026-02-12T09:14:21.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 228 16 59620 59604 1.02109 0.847656 0.0466292 0.0611846 2026-02-12T09:14:22.474 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 229 16 59827 59811 1.02017 0.808594 0.137893 0.0612483 2026-02-12T09:14:23.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 230 16 60036 60020 1.01928 0.816406 0.0519599 0.0613016 2026-02-12T09:14:24.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 231 16 60246 60230 1.01842 0.820312 0.0659191 0.0613585 2026-02-12T09:14:25.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 232 16 60443 60427 1.01735 0.769531 0.0660447 0.0614176 2026-02-12T09:14:26.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 233 16 60649 60633 1.01643 0.804688 0.05437 0.0614759 2026-02-12T09:14:27.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 234 16 60849 60833 1.01543 0.78125 0.23538 0.061536 2026-02-12T09:14:28.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 235 16 61062 61046 1.01465 0.832031 0.0860285 0.0615849 2026-02-12T09:14:29.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 236 16 61267 61251 1.01374 0.800781 0.0675795 0.0616392 2026-02-12T09:14:30.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 237 16 61475 61459 1.01289 0.8125 0.0493832 0.0616867 2026-02-12T09:14:31.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 238 16 61691 61675 1.01218 0.84375 0.0724371 0.0617332 2026-02-12T09:14:32.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 239 16 61900 61884 1.01136 0.816406 0.0960485 0.0617817 2026-02-12T09:14:33.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:14:33.475998+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0618259 2026-02-12T09:14:33.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:14:33.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 240 16 62116 62100 1.01066 0.84375 0.107705 0.0618259 2026-02-12T09:14:34.475 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 241 16 62318 62302 1.00974 0.789062 0.0509247 0.0618848 2026-02-12T09:14:35.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 242 16 62525 62509 1.00891 0.808594 0.0749561 0.0619337 2026-02-12T09:14:36.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 243 16 62738 62722 1.00818 0.832031 0.0396827 0.0619733 2026-02-12T09:14:37.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 244 16 62960 62944 1.0076 0.867188 0.0509033 0.0620133 2026-02-12T09:14:38.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 245 16 63176 63160 1.00694 0.84375 0.0712023 0.0620563 2026-02-12T09:14:39.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 246 16 63381 63365 1.0061 0.800781 0.0378638 0.0621053 2026-02-12T09:14:40.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 247 16 63581 63565 1.00519 0.78125 0.051523 0.0621599 2026-02-12T09:14:41.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 248 16 63794 63778 1.00449 0.832031 0.0704678 0.062207 2026-02-12T09:14:42.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 249 16 63997 63981 1.00364 0.792969 0.10852 0.062259 2026-02-12T09:14:43.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 250 16 64214 64198 1.00301 0.847656 0.0595048 0.0622968 2026-02-12T09:14:44.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 251 16 64429 64413 1.00236 0.839844 0.0662365 0.062333 2026-02-12T09:14:45.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 252 16 64617 64601 1.0013 0.734375 0.0480373 0.0623814 2026-02-12T09:14:46.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 253 16 64806 64790 1.00026 0.738281 0.106837 0.0624701 2026-02-12T09:14:47.476 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 254 16 65013 64997 0.999505 0.808594 0.0509795 0.0625165 2026-02-12T09:14:48.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 255 16 65225 65209 0.998833 0.828125 0.0404251 0.0625595 2026-02-12T09:14:49.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 256 16 65431 65415 0.998074 0.804688 0.0753318 0.0626062 2026-02-12T09:14:50.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 257 16 65622 65606 0.997094 0.746094 0.145619 0.0626679 2026-02-12T09:14:51.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 258 16 65820 65804 0.996227 0.773438 0.0709502 0.0627226 2026-02-12T09:14:52.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 259 16 66026 66010 0.995487 0.804688 0.0764765 0.0627724 2026-02-12T09:14:53.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:14:53.477544+0000 min lat: 0.0103456 max lat: 0.427899 avg lat: 0.0628187 2026-02-12T09:14:53.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:14:53.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 260 16 66227 66211 0.994678 0.785156 0.132591 0.0628187 2026-02-12T09:14:54.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 261 16 66434 66418 0.993964 0.808594 0.0563976 0.062862 2026-02-12T09:14:55.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 262 16 66649 66633 0.993376 0.839844 0.0831965 0.0628991 2026-02-12T09:14:56.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 263 16 66843 66827 0.99248 0.757812 0.0535356 0.0629352 2026-02-12T09:14:57.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 264 16 67011 66995 0.991206 0.65625 0.117751 0.0630434 2026-02-12T09:14:58.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 265 16 67207 67191 0.990355 0.765625 0.0371471 0.0630932 2026-02-12T09:14:59.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 266 16 67416 67400 0.989701 0.816406 0.0497231 0.0631392 2026-02-12T09:15:00.477 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 267 15 67604 67589 0.988759 0.738281 0.108204 0.0631958 2026-02-12T09:15:01.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 268 16 67812 67796 0.988086 0.808594 0.0605147 0.0632361 2026-02-12T09:15:02.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 269 16 68000 67984 0.987143 0.734375 0.0477929 0.0632892 2026-02-12T09:15:03.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 270 16 68193 68177 0.986279 0.753906 0.0607426 0.0633544 2026-02-12T09:15:04.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 271 16 68398 68382 0.985594 0.800781 0.146597 0.0633982 2026-02-12T09:15:05.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 272 16 68587 68571 0.984685 0.738281 0.119733 0.0634577 2026-02-12T09:15:06.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 273 16 68796 68780 0.984068 0.816406 0.12477 0.0634987 2026-02-12T09:15:07.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 274 16 69002 68986 0.983413 0.804688 0.0719224 0.0635396 2026-02-12T09:15:08.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 275 16 69197 69181 0.982607 0.761719 0.0722252 0.0635919 2026-02-12T09:15:09.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 276 16 69389 69373 0.981764 0.75 0.104788 0.0636431 2026-02-12T09:15:10.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 277 16 69599 69583 0.981181 0.820312 0.0510434 0.0636885 2026-02-12T09:15:11.478 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 278 16 69792 69776 0.980363 0.753906 0.0527257 0.0637321 2026-02-12T09:15:12.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 279 16 69992 69976 0.979649 0.78125 0.0937102 0.0637826 2026-02-12T09:15:13.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:15:13.479081+0000 min lat: 0.0103456 max lat: 0.489932 avg lat: 0.0638331 2026-02-12T09:15:13.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:15:13.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 280 16 70184 70168 0.978829 0.75 0.0809823 0.0638331 2026-02-12T09:15:14.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 281 16 70385 70369 0.978139 0.785156 0.0630326 0.0638823 2026-02-12T09:15:15.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 282 16 70584 70568 0.977427 0.777344 0.0419817 0.06393 2026-02-12T09:15:16.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 283 16 70784 70768 0.976734 0.78125 0.0623216 0.0639755 2026-02-12T09:15:17.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 284 16 70991 70975 0.976141 0.808594 0.0620757 0.0640119 2026-02-12T09:15:18.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 285 16 71190 71174 0.975444 0.777344 0.132677 0.0640604 2026-02-12T09:15:19.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 286 16 71370 71354 0.974491 0.703125 0.0466843 0.0641174 2026-02-12T09:15:20.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 287 16 71551 71535 0.973559 0.707031 0.0380792 0.0641804 2026-02-12T09:15:21.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 288 16 71751 71735 0.972891 0.78125 0.0928722 0.0642254 2026-02-12T09:15:22.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 289 16 71947 71931 0.972174 0.765625 0.0550256 0.0642723 2026-02-12T09:15:23.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 290 16 72158 72142 0.971663 0.824219 0.195266 0.0643081 2026-02-12T09:15:24.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 291 16 72357 72341 0.970995 0.777344 0.0869442 0.0643553 2026-02-12T09:15:25.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 292 16 72542 72526 0.970145 0.722656 0.152997 0.0644081 2026-02-12T09:15:26.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 293 16 72744 72728 0.969527 0.789062 0.0779501 0.064451 2026-02-12T09:15:27.479 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 294 16 72945 72929 0.968899 0.785156 0.144829 0.0644945 2026-02-12T09:15:28.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 295 16 73136 73120 0.968144 0.746094 0.0692624 0.0645434 2026-02-12T09:15:29.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 296 16 73318 73302 0.967275 0.710938 0.0533488 0.0645997 2026-02-12T09:15:30.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 297 16 73508 73492 0.966517 0.742188 0.0663481 0.0646513 2026-02-12T09:15:31.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 298 16 73711 73695 0.965934 0.792969 0.0676349 0.0646916 2026-02-12T09:15:32.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 299 16 73880 73864 0.964911 0.660156 0.0789079 0.0647522 2026-02-12T09:15:33.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout:2026-02-12T09:15:33.480615+0000 min lat: 0.0103456 max lat: 0.489932 avg lat: 0.0648054 2026-02-12T09:15:33.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T09:15:33.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout: 300 14 74067 74053 0.964156 0.738281 0.0292658 0.0648054 2026-02-12T09:15:34.480 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Total time run: 300.074 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Total writes made: 74067 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Write size: 4096 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Object size: 4096 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Bandwidth (MB/sec): 0.964176 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Stddev Bandwidth: 0.184322 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Max bandwidth (MB/sec): 1.46875 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Min bandwidth (MB/sec): 0.65625 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Average IOPS: 246 2026-02-12T09:15:34.481 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Stddev IOPS: 47.1864 2026-02-12T09:15:34.482 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Max IOPS: 376 2026-02-12T09:15:34.482 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Min IOPS: 168 2026-02-12T09:15:34.482 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Average Latency(s): 0.0648113 2026-02-12T09:15:34.482 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Stddev Latency(s): 0.034748 2026-02-12T09:15:34.482 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Max latency(s): 0.489932 2026-02-12T09:15:34.482 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Min latency(s): 0.0103456 2026-02-12T09:15:34.507 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Cleaning up (deleting benchmark objects) 2026-02-12T09:22:57.183 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Removed 74067 objects 2026-02-12T09:22:57.183 INFO:tasks.radosbench.radosbench.0.trial057.stdout:Clean up completed and total clean up time :442.676 2026-02-12T09:22:57.747 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2026-02-12T09:22:57.748 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2026-02-12T09:22:57.831 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:22:57.826+0000 7f566721a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:22:57.832 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:22:57.831+0000 7f566721a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:22:58.245 INFO:teuthology.orchestra.run.trial057.stderr:pool 'unique_pool_0' removed 2026-02-12T09:22:58.258 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-12T09:22:58.273 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-12T09:22:58.273 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-12T09:22:58.274 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial057.front.sepia.ceph.com 2026-02-12T09:22:58.274 DEBUG:teuthology.orchestra.run.trial057:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-12T09:22:58.291 DEBUG:teuthology.orchestra.run.trial057:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-12T09:22:58.345 DEBUG:teuthology.orchestra.run.trial057:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-12T09:22:58.404 INFO:teuthology.orchestra.run.trial057.stdout:"600" 2026-02-12T09:22:58.404 DEBUG:teuthology.orchestra.run.trial057:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-12_09:10:30:806494 /home/ubuntu/.ssh/authorized_keys 2026-02-12T09:22:58.461 DEBUG:teuthology.orchestra.run.trial057:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-12T09:22:58.525 DEBUG:teuthology.orchestra.run.trial057:> rm /home/ubuntu/.ssh/config 2026-02-12T09:22:58.591 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-12T09:22:58.606 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-12T09:22:58.722 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:22:58.717+0000 7fa58becb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:22:58.724 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-12T09:22:58.724+0000 7fa58becb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:22:58.864 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-12T09:22:58.864 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-12T09:22:58.865 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-12T09:22:58.865 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:22:58.965 INFO:tasks.ceph.osd.0:Stopped 2026-02-12T09:22:58.965 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-12T09:22:58.965 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:22:59.054 INFO:tasks.ceph.osd.1:Stopped 2026-02-12T09:22:59.054 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-12T09:22:59.055 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:22:59.149 INFO:tasks.ceph.osd.2:Stopped 2026-02-12T09:22:59.149 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-12T09:22:59.150 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-12T09:22:59.150 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:22:59.183 INFO:tasks.ceph.mgr.x:Stopped 2026-02-12T09:22:59.183 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-12T09:22:59.183 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-12T09:22:59.183 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:22:59.233 INFO:tasks.ceph.mon.a:Stopped 2026-02-12T09:22:59.233 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-12T09:22:59.233 DEBUG:teuthology.orchestra.run.trial057:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-12T09:22:59.298 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:22:59.299 DEBUG:teuthology.orchestra.run.trial057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-12T09:22:59.676 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:22:59.677 DEBUG:teuthology.orchestra.run.trial057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-12T09:22:59.707 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:22:59.707 DEBUG:teuthology.orchestra.run.trial057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-12T09:22:59.778 INFO:tasks.ceph:Archiving mon data... 2026-02-12T09:22:59.779 DEBUG:teuthology.misc:Transferring archived files from trial057:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41601/data/mon.a.tgz 2026-02-12T09:22:59.780 DEBUG:teuthology.orchestra.run.trial057:> mktemp 2026-02-12T09:22:59.834 INFO:teuthology.orchestra.run.trial057.stdout:/tmp/tmp.AMBcllYosP 2026-02-12T09:22:59.835 DEBUG:teuthology.orchestra.run.trial057:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.AMBcllYosP 2026-02-12T09:22:59.923 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod 0666 /tmp/tmp.AMBcllYosP 2026-02-12T09:23:00.004 DEBUG:teuthology.orchestra.remote:trial057:/tmp/tmp.AMBcllYosP is 198KB 2026-02-12T09:23:00.051 DEBUG:teuthology.orchestra.run.trial057:> rm -fr /tmp/tmp.AMBcllYosP 2026-02-12T09:23:00.065 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-12T09:23:00.065 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-12T09:23:00.186 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-12T09:23:00.186 INFO:tasks.ceph:Archiving crash dumps... 2026-02-12T09:23:00.188 DEBUG:teuthology.misc:Transferring archived files from trial057:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41601/remote/trial057/crash 2026-02-12T09:23:00.190 DEBUG:teuthology.orchestra.run.trial057:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-12T09:23:00.249 INFO:tasks.ceph:Compressing logs... 2026-02-12T09:23:00.249 DEBUG:teuthology.orchestra.run.trial057:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T09:23:00.273 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-12T09:23:00.273 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23618.log 2026-02-12T09:23:00.273 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-12T09:23:00.273 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22650.log 2026-02-12T09:23:00.274 INFO:teuthology.orchestra.run.trial057.stderr: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.23618.log.gz 2026-02-12T09:23:00.274 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-12T09:23:00.274 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23424.log 2026-02-12T09:23:00.274 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22650.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22650.log.gz 2026-02-12T09:23:00.274 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21976.log 2026-02-12T09:23:00.274 INFO:teuthology.orchestra.run.trial057.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-12T09:23:00.274 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23882.log 2026-02-12T09:23:00.275 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23424.log.gz 2026-02-12T09:23:00.275 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.21976.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.21976.log.gz 2026-02-12T09:23:00.275 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23280.log 2026-02-12T09:23:00.275 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22160.log 2026-02-12T09:23:00.275 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23882.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23882.log.gz 2026-02-12T09:23:00.275 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22883.log 2026-02-12T09:23:00.275 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23280.log.gz 2026-02-12T09:23:00.276 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22981.log 2026-02-12T09:23:00.276 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22160.log.gz 2026-02-12T09:23:00.276 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23569.log 2026-02-12T09:23:00.276 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22883.log.gz 2026-02-12T09:23:00.276 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23133.log 2026-02-12T09:23:00.276 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22981.log.gz 2026-02-12T09:23:00.276 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22254.log 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23569.log.gz 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23426.log 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23133.log.gz 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23427.log 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.22254.log.gz 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24057.log 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23426.log.gz 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23131.log 49.6% 2026-02-12T09:23:00.277 INFO:teuthology.orchestra.run.trial057.stderr: -- replaced with /var/log/ceph/ceph-client.admin.23427.log.gz 2026-02-12T09:23:00.278 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.24057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23905.log 2026-02-12T09:23:00.278 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.24057.log.gz 2026-02-12T09:23:00.278 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22793.log 2026-02-12T09:23:00.278 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23131.log.gz 2026-02-12T09:23:00.278 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.0.23905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23030.log 2026-02-12T09:23:00.278 INFO:teuthology.orchestra.run.trial057.stderr: 66.4% -- replaced with /var/log/ceph/ceph-client.0.23905.log.gz 2026-02-12T09:23:00.278 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21881.log 2026-02-12T09:23:00.278 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22793.log.gz 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22207.log 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.23030.log.gz 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.21881.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.21881.log.gz 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20598.log 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22023.log 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22207.log.gz 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph.tmp-client.admin.20598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23279.log 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr: 68.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20598.log.gz 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24242.log 2026-02-12T09:23:00.279 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22023.log.gz 2026-02-12T09:23:00.280 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23279.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-12T09:23:00.280 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23279.log.gz 2026-02-12T09:23:00.280 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.24242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23520.log 2026-02-12T09:23:00.280 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.24242.log.gz 2026-02-12T09:23:00.280 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-12T09:23:00.280 INFO:teuthology.orchestra.run.trial057.stderr: 86.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-12T09:23:00.280 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22503.log 2026-02-12T09:23:00.280 INFO:teuthology.orchestra.run.trial057.stderr: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.23520.log.gz 2026-02-12T09:23:00.281 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22932.log 2026-02-12T09:23:00.281 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23278.log 2026-02-12T09:23:00.281 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22503.log.gz 2026-02-12T09:23:00.281 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23829.log 2026-02-12T09:23:00.281 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22932.log.gz 2026-02-12T09:23:00.281 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23278.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23278.log.gz 2026-02-12T09:23:00.281 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-12T09:23:00.282 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23132.log 2026-02-12T09:23:00.282 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23829.log: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.23829.log.gz 2026-02-12T09:23:00.282 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-12T09:23:00.282 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.23132.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23132.log.gz 2026-02-12T09:23:00.282 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22818.log 2026-02-12T09:23:00.282 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22723.log 2026-02-12T09:23:00.282 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22473.log 2026-02-12T09:23:00.283 INFO:teuthology.orchestra.run.trial057.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.22818.log.gz 2026-02-12T09:23:00.283 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22723.log: 88.0% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-12T09:23:00.283 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.22723.log.gz 2026-02-12T09:23:00.283 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.22473.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22473.log.gz 2026-02-12T09:23:00.301 INFO:teuthology.orchestra.run.trial057.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-12T09:23:44.157 INFO:teuthology.orchestra.run.trial057.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-12T09:23:45.097 INFO:teuthology.orchestra.run.trial057.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-12T09:23:54.107 INFO:teuthology.orchestra.run.trial057.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-12T09:23:54.108 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-12T09:23:54.108 INFO:teuthology.orchestra.run.trial057.stderr:real 0m53.844s 2026-02-12T09:23:54.108 INFO:teuthology.orchestra.run.trial057.stderr:user 2m16.163s 2026-02-12T09:23:54.109 INFO:teuthology.orchestra.run.trial057.stderr:sys 0m6.118s 2026-02-12T09:23:54.110 INFO:tasks.ceph:Archiving logs... 2026-02-12T09:23:54.110 DEBUG:teuthology.misc:Transferring archived files from trial057:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41601/remote/trial057/log 2026-02-12T09:23:54.111 DEBUG:teuthology.orchestra.run.trial057:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-12T09:24:04.686 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T09:24:04.759 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T09:24:04.769 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-12T09:24:04.769 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-12T09:24:04.795 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-12T09:24:04.806 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-12T09:24:04.806 DEBUG:teuthology.orchestra.run.trial057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T09:24:04.850 INFO:teuthology.orchestra.run.trial057.stderr:bash: line 1: ntpq: command not found 2026-02-12T09:24:09.875 INFO:teuthology.orchestra.run.trial057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T09:24:09.875 INFO:teuthology.orchestra.run.trial057.stdout:=============================================================================== 2026-02-12T09:24:09.876 INFO:teuthology.orchestra.run.trial057.stdout:^+ clock.nyc.he.net 3 6 377 4 -5778us[-5778us] +/- 69ms 2026-02-12T09:24:09.876 INFO:teuthology.orchestra.run.trial057.stdout:^+ 172-235-154-118.ip.linod> 4 7 377 76 -1891us[-1927us] +/- 56ms 2026-02-12T09:24:09.876 INFO:teuthology.orchestra.run.trial057.stdout:^+ owners.kjsl.com 2 7 377 75 +2308us[+2272us] +/- 39ms 2026-02-12T09:24:09.876 INFO:teuthology.orchestra.run.trial057.stdout:^* 96-19-94-82.cpe.sparklig> 1 7 377 74 +1230us[+1193us] +/- 32ms 2026-02-12T09:24:09.877 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-12T09:24:09.894 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-12T09:24:09.895 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-12T09:24:09.912 DEBUG:teuthology.orchestra.run.trial057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-12T09:24:09.963 DEBUG:teuthology.orchestra.run.trial057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T09:24:10.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T09:24:10.028 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-12T09:24:10.047 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-12T09:24:10.063 INFO:teuthology.task.internal:Duration was 1117.786979 seconds 2026-02-12T09:24:10.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-12T09:24:10.080 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-12T09:24:10.081 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-12T09:24:10.117 INFO:teuthology.orchestra.run.trial057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T09:24:10.277 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-12T09:24:10.278 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial057.front.sepia.ceph.com 2026-02-12T09:24:10.278 DEBUG:teuthology.orchestra.run.trial057:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-12T09:24:10.336 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-12T09:24:10.337 DEBUG:teuthology.orchestra.run.trial057:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T09:24:10.419 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-12T09:24:10.419 DEBUG:teuthology.orchestra.run.trial057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T09:24:10.482 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T09:24:10.483 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T09:24:10.484 INFO:teuthology.orchestra.run.trial057.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-12T09:24:10.484 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T09:24:10.484 INFO:teuthology.orchestra.run.trial057.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-12T09:24:10.487 INFO:teuthology.orchestra.run.trial057.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 87.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-12T09:24:10.489 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-12T09:24:10.506 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-12T09:24:10.506 DEBUG:teuthology.orchestra.run.trial057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-12T09:24:10.554 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-12T09:24:10.571 DEBUG:teuthology.orchestra.run.trial057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-12T09:24:10.618 INFO:teuthology.orchestra.run.trial057.stdout:kernel.core_pattern = core 2026-02-12T09:24:10.632 DEBUG:teuthology.orchestra.run.trial057:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-12T09:24:10.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T09:24:10.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-12T09:24:10.704 INFO:teuthology.task.internal:Transferring archived files... 2026-02-12T09:24:10.705 DEBUG:teuthology.misc:Transferring archived files from trial057:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41601/remote/trial057 2026-02-12T09:24:10.705 DEBUG:teuthology.orchestra.run.trial057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-12T09:24:10.761 INFO:teuthology.task.internal:Removing archive directory... 2026-02-12T09:24:10.761 DEBUG:teuthology.orchestra.run.trial057:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-12T09:24:10.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-12T09:24:10.829 INFO:teuthology.task.internal:Not uploading archives. 2026-02-12T09:24:10.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-12T09:24:10.849 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-12T09:24:10.849 DEBUG:teuthology.orchestra.run.trial057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-12T09:24:10.867 INFO:teuthology.orchestra.run.trial057.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 12 09:24 /home/ubuntu/cephtest 2026-02-12T09:24:10.869 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-12T09:24:10.887 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-12T09:24:10.928 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/radosbench_omap_write} duration: 1117.7869791984558 flavor: debug owner: scheduled_matan@soko04.front.sepia.ceph.com success: true 2026-02-12T09:24:10.928 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T09:24:11.056 INFO:teuthology.run:pass